[ExI] Why do the language model and the vision model align?

[John Clark]

On Tue, Feb 10, 2026 at 9:21 AM Jason Resch via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

*>> What's surprising to me is that an AI that was only trained on words
>> can converge on ANYTHING, let alone to something congruent to the real
>> world. If we had access to an extra terrestrial's library that was huge but
>> contained no pictures, just 26 different types of squiggles arranged into
>> many trillions of words, I don't see how we could ever make any sense out
>> of it, I don't see how we could ever write a new sentence in ET's language
>> that was not only grammatically correct but also expressed an idea that was
>> true and non-trivial; but somehow an AI could. Don't ask me how.*
>>
>
> *> An ideal compression of Wikipedia would require not only understanding
> all the grammar of our languages, but also how our minds think, and what
> our world looks like and how it behaves. Our base 10 numerical
> representation system would quickly fall to such compression, given all the
> numbered lists that so commonly appear across the site.*
>

*In general it's impossible to prove that you're using the ideal
compression algorithm on your data, for all you know there might be a
way to compress it even more. OK, I can see how you might be able to
determine that a book written by ET was about arithmetic, and that some of
the symbols represented integers and not letters, and that a base 10 system
was used. But I don't see how you could use the same procedure to determine
that another book was about chemistry unless you already had a deep
understanding of how real world chemistry worked, and I don't see how you
could obtain such chemical knowledge without experimentation and by just
looking at sequences of squiggles. But apparently there is a way. *

*> The periodic table could then be understood as elements containing
> different numbers of fundamental particles.*


*The existence of isotopes would greatly complicate things, for example we
know that the element Tin has 10 stable isotopes and 32 radioactive ones,
they all have identical chemical properties but, because of neutrons, they
all have different masses. I don't see how you could ever deduce that fact
without experimentation if you started with nothing but a knowledge of
arithmetic. We obtained our knowledge of chemistry through experimentation
but how did an AI, which has never observed anything except sequences of
squiggles, obtain such knowledge?   *


*> If there's an article on the "triple alpha process" the ideal
> compression algorithm would know that "carbon" is the most likely word that
> follows "three helium nuclei can combine to yield ".*


*To be able to predict that the three alpha process produces carbon, or
even be able to predict that something called an "alpha particle" exists,
you'd need to already have a deep understanding of a thing as unintuitive
as Quantum Mechanics, and that would be even more difficult to obtain from
first principles than knowledge of chemistry.  Perhaps Mr. Jupiter
Brain could deduce the existence of the physical world starting from
nothing but arithmetic (but I doubt it) however it is certainly far far
beyond the capability of any existing AI, so they must be using some other
method. I just wish I knew what it was. *

*> To compress perfectly is to understand perfectly.*


*But perfection is not possible. In general, finding the "perfect"
compression, the absolute shortest representation of a piece of data, is an
uncomputable problem. By chance you might be using the best possible
compression algorithm on your data, but there's no way to prove to yourself
or to anybody else that you are.*

*  John K Clark*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260211/1bfd1f38/attachment.htm>