[ExI] Why do the language model and the vision model align?

[Keith Henson]

If text and visual AI-trained models failed to converge, it would
surprise the heck out of me.

They are both based on reality (whatever that is).

Keith

On Mon, Feb 23, 2026 at 6:55 AM John Clark via extropy-chat
<extropy-chat at lists.extropy.org> wrote:
>
> On Sun, Feb 22, 2026 at 9:38 AM Jason Resch via extropy-chat <extropy-chat at lists.extropy.org> wrote:
>
>>> >> No, you STILL haven't answered my question. Which is more fundamental, the English word  "c-o-w" or the thing with four legs that can produce milk? I think the thing with four legs is more fundamental. Do you disagree?
>>
>>
>> > The cow just more fundamental than our ideas or language describing cows.
>
>
> Thank you.
>
>> > Just as the physical universe is more fundamental than our human ideas and theories about physics. And just as the integers are more fundamental than our symbols like "1" and "2".
>
>
> So we both agree that the underlying mathematics would be the same regardless of what language was used to express it, but do you also believe, as most mathematicians do, that mathematics itself is a language?  I think it is and if it is then like any language it could be used to write both fiction and nonfiction, and some very abstract realms of higher mathematics might be compared to a Lord Of The Rings book telling us in great detail how dragons behave even though dragons do not exist and even describing other languages even though nothing has ever used or is using or will use those languages except the author of the book. In literature that's called "world building".
>
>>>>
>>>> >>> When I talk about the plausibility of mathematical objects as plausibly being fundamental I am speaking of layers 1 & 2, not layer 8.
>>>> 8. Human ideas about math and physics
>>>> 7. Human Ideas
>>>> 6. Human minds
>>>> 5. Human brains
>>>> 4. Our physical universe
>>>> 3. All existing universes in a multiverse
>>>> 2. All computations playing out in all possible ways
>>>> 1. Mathematical truth
>>>
>>>
>>> >> How can you speak about layers 1 & 2 when you only have access to layers 6,7 and 8?
>>
>>
>> > For the same reason we can speak of layer 3.
>
>
> I can speak of it because I don't believe in your 8 layers, but you do so you can't.
>
>> > The quantum mechanical properties of this universe strongly suggests it is part of (at least) a quantum multiverse. Likewise the inflationary conditions of the early universe strongly suggests it is part of an infinite eternally inflating universe among countless other bubbles. And similarly, the extreme fine tuning of forces and constants required for life to be possible strongly suggests our universe is one of a huge number of physical universes ruled by other laws.
>
>
> I agree with all of that. Physics is capable of producing a hell of a lot of interesting and very diverse stuff, but it cannot produce everything that is conceivable. In none of those other universes is the Second Law Of Thermodynamics untrue and, although this is a little less certain, probably in none of those universes will an electron spontaneously turn into a proton. Nevertheless it would be possible to write a story in English or Spanish or Mathematics or any language in which both those things occur.  You could even write a story in which 2+2=5, although such a story would not be consistent, it would be full of plot holes.
>
>>>
>>> >> And Information is physical, there is no way you can have "all computations playing out in all possible ways", or even compute 2+2, without a physical universe. You need hardware,
>>
>>
>> > Have you ever stopped to ask what hardware computes how an electron behaves in any given situation?
>
>
> Yes I have because I'm a retired electrical engineer and that's how they make their living.
>
>>
>> > Or consider a photon, which is physically trapped in 0-time, it can't change and therefore it can't compute anything on its own.
>
>
> It's true that time doesn't exist from the viewpoint of a photon, so because of that you could conclude that a photon can't compute anything, or you could conclude that a photon can compute everything. But I conclude that neither viewpoint is useful.
>
>> >  I would again ask what computational substrate powers all the 10^106 operations per second that occur within just the observable part of our universe?
>
>
> The answer is the laws of physics. For example the Pauli Exclusion Principle says that an electron in an atom can be in any quantum state but 2 electrons can NOT be in the same quantum state, so physics recognizes a difference between 1 and 2, and once that is done any integer can be defined and computations are possible. Incidentally the Pauli Exclusion Principle is the only reason that the chair you're sitting in right now does not sink down to the center of the Earth.
>
>
>> > My own answer to that question is "particular mathematical structures, by their very nature, can compute on their own."
>
>
> If mathematical structures can perform computations on their own then why is Nvidia the most valuable company in the world?  Instead of spending trillions of dollars on huge data centers why don't companies like OpenAI and Anthropic just buy a book about those data structures and let the book perform those computations? I'll tell you why, because that won't work.
>
>
>> > The discoveries are not mine, I simply am trying to get the word out by writing about them.
>> See the work of Marchal,  Standish, Mueller, and Wolfram
>
>
> Standish is a big fan of Marchal and I am VERY familiar with Marchal's work, and I find it to be utterly worthless. I don't know Mueller but I have read a few books by Wolfram and although I don't agree with everything he said I certainly wouldn't say his ideas are worthless.
>
>
>>
>> >  You have been on a mailing list with 2 out of 4 of these people for decades and you still pretend these ideas are alien to you.
>
>
> Far from pretending these ideas are unknown to me I've been debating with those 2 people for the better part of a decade, and I still think their ideas are full of holes.
>
>>  > Now for the big one: see Russell's work in deducing the Schrodinger equation from pure mathematics. Surely, you will admit, if that can be done,
>
>
> Deriving Schrodinger's Equation from pure mathematics means that Schrodinger's Equation is consistent with the laws of logic and therefore of mathematics. Well I should hope so!! There are an infinite number of things that are consistent with the laws of mathematics and Schrodinger's Equation happens to be one of them. But Schrodinger's equation is also consistent with the laws of physics, and that could NOT have been predicted from just the laws of mathematics.
>
>
>>> >> A muon is identical to an electron except that it's 207 times more massive and it only lives for 2 *10^-6 seconds before decaying, and a tau particle is identical to a muon except that it's 3477 times more massive than an electron and it only lives for 2.9*10*-13 seconds; and both the muon and the tau have their own special type of neutrino associated with them. As far as we know none of those 4 particles plays an important part in the operation of the universe; as one scientist famously asked "who ordered that?".
>>
>>
>> > But can these 3 energy states for half-spin particles be removed without making the laws themselves more complex?
>
>
> The answer is probably yes. It is a fact that the laws of physics were simpler before the muon and the tau were discovered, we had to make them more complex to account for these extremely rare things, and as far as we know the universe could've gotten along just fine without them.
>
>
>>> >> It also cannot explain why there is an arrow of time other than to say it's because the universe started out in a very low entropy state, but why it started out in such a low state today's physics can't say.
>>
>>
>> > There is another answer hardly anyone pays any attention to, which does not require the universe to begin in a low entropy state. It requires only that the maximum entropy state at a given period of time, increases more quickly than the universe can reach equilibrium with that maximum entropy state.
>
>
> First of all, before you can use a word like "quickly" in your answer you've got to explain why time has the fundamental properties that it has.  And it's easy to understand why tomorrow the universe will be in a higher entropy state than it is today, it's because tomorrow will be in a different  state than it is today (otherwise "today" and "tomorrow" would mean the same thing) and there are vastly more ways something can be disordered than ways than can be ordered.
>
> But by using the exact same logic you must conclude that yesterday the universe was in a higher entropy state than it is today, and that is not true. You need to add another axiom to explain why there is an arrow of time, one that cannot be deduced from pure mathematics, and it is "the universe started out in a low entropy state, lower than anything that has occurred since".
>
>>
>> > Note that this happens naturally in an expanding universe.
>
>
> That is one possible result of a universe starting out in a low entropy state.
>
>
>>>
>>> >> It is neither scientific nor unscientific, and it is not an attitude, it's simply acknowledging the undeniable logical FACT that it's impossible to provide an answer if you don't know the question.
>>
>>
>> > I have a clear list of questions. You think they're unanswerable,
>
>
> I don't know if those questions are answerable or not, but I do know that you wouldn't have found your list of questions if you hadn't had access to physical reality and knew nothing but pure mathematics.
>
>> > we can justify a belief in a mathematical reality empirically, by seeing what predictions such an ontology implies for our observations, and comparing those predictions against what we see.
>
>
> Quantum Mechanics wasn't discovered by somebody deriving it from pure mathematics, everybody was satisfied with Newtonian physics and Maxwell's equations until technology improved enough that we could perform experiments that we were unable to do before and we started to get some very weird results.  Max Planck, the guy who invented the quantum in 1900, said he did it in an act of desperation because it was the only thing that enabled him to make predictions, and even then he thought it was just a mathematical trick and did not indicate anything physical; it wasn't until Einstein's 1905 paper on the Photoelectric Effect (the thing that got him the Nobel prize) did it become clear that the quantum was physical and not just a mathematical artifact.
>
>   John K Clark
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>>>>>>> Also, consider a mathematical model of a hurricane and a real physical hurricane, is the physical hurricane modeling the mathematical representation or is the mathematical representation modeling the physical hurricane?  You'd expect the real deal to be more complex than a mere model, so if you're right then the physical hurricane should be simpler than the mathematical model that is running on a computer, but that is not the case. It is never the case, the mathematical model always uses approximations, the physical hurricane never does.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>
>>>>>>>>> Even large finite numbers can't exist in our heads. Computers have calculated 105 trillion digits of π, but if you want to calculate the circumference of the observable universe from its radius to the greatest accuracy that physically makes sense, the Planck length, you'd only need the first 62 digits. So I think the 63rd digit has less reality than the 62nd, and the 105 trillionth even less.
>
>
>
>>
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat