[ExI] Why do the language model and the vision model align?

[Jason Resch]

On Tue, Feb 24, 2026 at 11:29 AM John Clark <johnkclark at gmail.com> wrote:

> On Mon, Feb 23, 2026 at 11:49 AM Jason Resch <jasonresch at gmail.com> wrote:
>
>>
> *> We must take care to disambiguate here. The string "mathematics" can be
>> used in one sense to refer the human invented language, or it can be used
>> to refer to the objects which mathematicians study. For you, who take a
>> non-platonist position, you have no room in your ontology for this latter
>> sense of the word.*
>>
>
> *No, that is not true, there will always be a difference between what we
> know about mathematics and what is true about mathematics. Godel discovered
> that some things are true but have no proof, that is to say there's no way
> to produce those true statements in a finite number of steps from the
> Zermelo–Fraenkel axioms plus the Axiom Of Choice. And then just a few years
> later Turing discovered that in general there's no way to tell if a
> mathematical statement is true but unprovable or is just false. If
> Goldbach's Conjecture is one of these (and if it isn't there are infinite
> number of similar conjectures that are) then a million years from now even
> Mr. Jupiter Brain will be trying, unsuccessfully, to find a proof in order
> to prove that it is correct, and will still be grinding through huge
> numbers trying, unsuccessfully, to find a counterexample, an even number
> that is not the sum of two prime numbers, to prove it is false. *
>
> *So if you want to postulate some sort of platonic heaven where everything
> that is true resides then you can, but it seems rather pointless because we
> can never enter that heaven or have any contact with it.*
>

No such "place" is needed to hold such truths. On this we agree. All that
is needed for the result I describe to follow is for these truths to exist
independently of us (and here you seem to agree with that point, there
being a difference betwwen mathematical truth, and what we humans can
know/prove/access of that truth).



> * What's more it cannot be what's responsible for computation because
> computation needs change and everything that is true can not change. But
> our knowledge of what is true can change, and so can a physical system.*
>


Are you familiar with the block universe view that emerges from Einstein's
relativity? Have you heard Tegmark's description of the universe (from the
bird's eye view) looking like a video tape, while from the frog's eye view
of those inside the universe, they see only one frame at a time? I think
the only way to reconcile these two consistent views of time and the
universe, is to recognize time to be a subjective phenomenon, much like the
branching structure of the wave function under many worlds creates the
*appearance of collapse* even when there is *objectivcely* no collapse, the
nature of our brain's, and how they process and store and remember
information along one direction in the arrow of time, creates the
*appearance of a flow of time* even though there is *objectively* no flow
of time. This upgraded understanding of time, in seeing the objective vs.
subjective differences, is in my opinion a requirement before anyone can
view physical universes as static mathematical objects, or as consequences
of eternal unchanging mathematical truths.

Change is possible in mathematical objects, or universes, or computational
functions, but change is always in respect to something. Think of a plot of
a graph of y=f(x) on an X-Y coordinate plane. The entire graph is static,
and yet, we can say that f(x) changes with respect to x. Now consider a
particle embedded in the four dimensional block-time conception of our
universe. We can still say that the position p(t) of the particle, changes
in respect to time t. Or consider the execution of a program, we can say
the state of the memory of the Turing machine changes in respect to the
number of steps the Turing machine has performed.


>
>
>> *> You are a fan of Max Tegmark's definition of consciousness being "how
>> information feels when it is being processed."*
>>
>
> *True.*
>
> *> Have you read Tegmark's book "Our Mathematical Universe"?*
>>
>
> *Yes I have and it's a very good book well worth reading, but near
> the end I think he goes one step too far. Tegmark wrote an even better book
> 3 years later "Life 3.0: Being Human in the Age of Artificial Intelligence"*
> *.*
>
>
> *>>> Have you ever stopped to ask what hardware computes how an electron
>>>> behaves in any given situation?*
>>>>
>>>
>>> *>>Yes I have because I'm a retired electrical engineer and that's how
>>> they make their living. *
>>>
>>
>> *> Did you ever find a satisfactory answer?*
>>
>
> *Yes I have and I've already mentioned it, perhaps you missed it so I'll
> repeat it here: *
> *"The answer is the laws of physics. For example the Pauli Exclusion
> Principle says that an electron in an atom can be in any quantum state but
> 2 electrons can NOT be in the same quantum state, so physics recognizes a
> difference between 1 and 2, and once that is done any integer can be
> defined and computations are possible. Incidentally the Pauli Exclusion
> Principle is the only reason that the chair you're sitting in right now
> does not sink down to the center of the Earth." *
>

But to use your own objection "*a law can't do **anything*" just as you
claimed "*a book can't do anything*." The problem here in both cases, lies
in confusing the "human-idea of a law," or "the human idea of a
mathematical object," with the *actual instantiation of physical laws*, or
the *actual instantiation of the mathematical object*, respectively.


>
>
>
>> *> how can a quantum computer (that could also fit on your desk) compute
>> over 2^10,000 computational states, when there aren't even 2^100 particles
>> in the entire observable universe?*
>>
>
> *N particles can be in far more than N states, for example chess pieces
> can only be in 64 different places but there are about 10^120 different
> chess games, assuming that in a game each player moves a piece about 40
> times. And yet way back in 1997 a conventional computer, that was very
> primitive by today's standards, was able to beat the best human chess
> grandmaster in the world at the game. *
>

But does this not suggest to you (as it does to me) that there is
something *much
greater than the physical universe* underlying it all?


>
>
>> *> As Feynman wondered:*
>> *"It always bothers me that, according to the laws as we understand them
>> today, it takes a computing machine an infinite number of logical
>> operations to figure out what goes on in no matter how tiny a region of
>> space, and no matter how tiny a region of time. How can all that be going
>> on in that tiny space? Why should it take an infinite amount of logic to
>> figure out what one tiny piece of space/time is going to do?"*
>> -- Richard Feynman in “The Character of Physical Law” (1965)
>>
>
> *And Feynman was able to give a partial answer to his question:  *
>
> * "Nature isn't classical, dammit, and if you want to make a simulation of
> nature, you'd better make it quantum mechanical, and by golly it's a
> wonderful problem because it doesn't look so easy." *
>
> *He was right, it wasn't easy, but quantum computers look a lot easier now
> than they did in Feynman's day. *
>

Of course he's right, but simply saying "it's quantum" does nothing to
resolve the mystery of why nature is this way.


>
> *>> If mathematical structures can perform computations on their own then
>>> why is Nvidia the most valuable company in the world?  Instead of spending
>>> trillions of dollars on huge data centers why don't companies like OpenAI
>>> and Anthropic just buy a book about those data structures and let the book
>>> perform those computations? I'll tell you why, because that won't work. *
>>>
>>
>> *> You believe this universe's "laws of physics" can compute on their
>> own. You also believe in other universes, which also, presumably can
>> compute on their own.*
>>
>
> *Yes.*
>
>
> *> These other universes are mathematical structures that compute on their
>> own. *
>>
>
> *No. Mathematical structures can't compute because mathematical structures
> can't change, but physical structures can change and so they can compute. *
>


See my earlier answer to this.


>
>
> *>> Standish is a big fan of Marchal and I am VERY familiar with Marchal's
>>> work, and I find it to be utterly worthless. *
>>>
>>
>> > I don't think you ever understood either of them. You spend years
>> arguing over what first-person and third-person meant, and never considered
>> any of the meat of his argument.
>>
>
> *Marchal was the one who kept talking about the transcendental importance
> of the difference between first person and third person not me, and in his
> "proof" about the nature of personal identity he kept on using words like
> "you" and "your" in thought experiments as if the meanings of those words
> were already clear. But that's what he was trying to prove! He was stating
> things in his "proof" that he was trying to prove, and that's why it was
> worthless. It contained nothing that was profound so understanding it was
> not difficult, but reading it without laughing was difficult.    *
>
>
>> *>> I don't know Mueller but I have read a few books by Wolfram and
>>> although I don't agree with everything he said I certainly wouldn't say his
>>> ideas are worthless. *
>>>
>>
>> *> I am glad you think Wolfram's ideas are not worthless.*
>> *Here is a quick introduction to Mueller's work in a presentation
>> form: https://www.youtube.com/watch?v=Tm3h_6UU2jY
>> <https://www.youtube.com/watch?v=Tm3h_6UU2jY>*
>> *Or if you prefer reading, this article covers the main
>> basis: https://arxiv.org/abs/1712.01826 <https://arxiv.org/abs/1712.01826>*
>>
>
> *Apparently Mueller is as silly as Marchal, like him the man believes that
> it is of profound significance that physics cannot give even a
> probabilistic answer to the question " if 2 perfect copies of you are made
> and one goes to Washington and one goes to Moscow  which city will you find
> yourself in?". Well of course physics can't give an answer to that because
> not every string of words that happens to have a question mark at the end
> is a question, sometimes it's just gibberish. How do I know this thought
> experiment is ridiculous? Because even after the experiment has been
> completed nobody can say what the correct answer should have been. It's
> amazing how good personal pronouns are at hiding nonsense, if instead of
> asking which city will you see Mueller and Marchal had asked which city
> will John Clark see then that would NOT have been nonsense, it would've had
> an answer, and the answer would have been "both". *
>

Add Tegmark to your list of silly people, for he says the exact same thing
in Our Mathematical Universe:

"It gradually hit me that this illusion of randomness business really
wasn’t specific to quantum mechanics at all. Suppose that some future
technology allows you to be cloned while you’re sleeping, and that your two
copies are placed in rooms numbered 0 and 1. When they wake up, they’ll
both feel that the room number they read is completely unpredictable and
random."
-- Max Tegmark in “Our Mathematical Universe” (2014)

So could it be that all these great thinkers, Marchal, Muller, Tegmark, are
just plain silly, or might it be that they are on to something significantm
which for some reason you simply aren't appreciating?



>
>
>
>> *>  In his 2004 paper and 2006 book, Russell Standish showed he could
>> derive theree postulates of quantum mechanics, including the Schrödinger
>> equation, purely from basic assumptions about observation*
>>
>
>
> *Standish demonstrates a keen grasp of the obvious!  Of course
> Schrödinger's equation can be deduced from observation, historically that
> is exactly how it was found. But nobody would have proposed such a crazy
> thing if the results of experiments hadn't demanded it. Yes it can be
> derived from pure mathematics, that is to say it has no mathematical
> errors, but an infinite number of equations can be derived from pure
> mathematics that contain no mathematical errors however very few of them
> have anything to do with physics and many of them have been experimentally
> proven to be wrong. *
>

The claim is not that Standish observed something in nature, and developed
the Schrödinger equation to explain those observations.
The claim is Standish made some basic assumptions about the nature of
observation, and then showed how one can, starting only from those
assumptions, derive the Schrödinger equation deductively, (not empirically).


>
> *>>> But can these 3 energy states for half-spin particles be removed
>>>> without making the laws themselves more complex?*
>>>>
>>>
>>> *>>The answer is probably yes. It is a fact that the laws of physics
>>> were simpler before the muon and the tau were discovered, we had to make
>>> them more complex to account for these extremely rare things, and as far as
>>> we know the universe could've gotten along just fine without them.    *
>>>
>>
>> *> The universe could have, but could life? *
>>
>
> *Probably. Maybe things will change tomorrow but nobody has yet provided a
> compelling reason why it could not.  *
>

Well if you can show that, then you can disprove this hypothesis of
Standish, Tegmark, and Mueller, and we need not debate it any more.



>
>
>
> *>>  it's easy to understand why tomorrow the universe will be in a higher
>>> entropy state than it is today, it's because tomorrow will be in
>>> a different  state than it is today (otherwise "today" and "tomorrow" would
>>> mean the same thing) and there are vastly more ways something can be
>>> disordered than ways than can be ordered.  **But by using the exact
>>> same logic you must conclude that yesterday the universe was in a higher
>>> entropy state than it is today, and that is not true. You need to add
>>> another axiom to explain why there is an arrow of time, one that cannot be
>>> deduced from pure mathematics, and it is "the universe started out in a low
>>> entropy state, lower than anything that has occurred since".*
>>>
>>
>> *> the early universe could have begun in a maximum entropy state, where
>> everything was at thermal equilibrium. However, due to the expansion of the
>> universe *[...]
>>
>
> *That is impossible. The universe couldn't have been born in a maximum
> entropy state because the expansion would cause the entropy to become even
> larger. The reason comes down to gravity, for a gas high entropy means that
> the gas is spread out evenly, but when gravity comes into the picture high
> entropy means that matter is clumped together, like in a Black Hole. In the
> early universe matter was spread out very evenly so it had very low
> gravitational entropy. If the universe had been born at maximum entropy, it
> would have started as a collection of black holes, not a smooth plasma.*
>

That's false. Black holes only have maximum entropy for a given volume.
They have less entropy than the same mass spread out over a greater volume.
This is clearly evident from Bekenstein's bound calculation. Maximum
entropy is proportional to (mass * volume). We've argued this before, you
have apparently not remembered anything from that last conversation. 1 kg
of matter in a microscopic black hole, has considerably less entropy than 1
kg worth of energy in the form of IR photons bouncing around inside a
sphere with a 1 meter radius. The more things are clumped together within a
given volume, the lower the entropy of that system is.


>
>
>> * > All that is needed to support an arrow of time is for the maximum
>> possible entropy for a given volume of the universe, to increase faster
>> than equilibrium can catch up.*
>>
>
> *Well yeah, the entropy of the universe has been increasing ever since the
> Big Bang but it has never caught up to the maximum possible limit. The gap
> between the actual entropy and the maximum possible entropy is what allows
> for complex structures like life to exist. *
>

Yes, but what is important to note (to escape the common misconception that
the universe had to begin in a special improbable state) is to recognize
that maximum entropy can also grow. Therefore the early state of the
universe need not have been low in relation to the (at the time) maximum
possible entropy. In fact, the early universe (say when it was a
quark-gluon plasma) was likely at or near a maximum entropy state (for that
epoch of the universe).


>
>
>
>> *>> Quantum Mechanics wasn't discovered by somebody deriving it from pure
>>> mathematics, everybody was satisfied with Newtonian physics and Maxwell's
>>> equations until technology improved enough that we could perform
>>> experiments that we were unable to do before and we started to get some
>>> very weird results.  Max Planck, the guy who invented the quantum in 1900,
>>> said he did it in an act of desperation because it was the only thing that
>>> enabled him to make predictions, and even then he thought it was just a
>>> mathematical trick and did not indicate anything physical; it wasn't until
>>> Einstein's 1905 paper on the Photoelectric Effect (the thing that got him
>>> the Nobel prize) did it become clear that the quantum was physical and not
>>> just a mathematical artifact. *
>>>
>>
>> *> What you describe above is an accident of history. Had computer
>> science progressed in Babbage's time rather than in Turing's time, it's
>> possible that the equivalent Marchal, Standish, Meuller, or Wolfram of
>> their time could have anticipated a quantum mechanical reality before
>> Planck's and Einstein found experimental evidence of it.*
>>
>
> *Even if Hugh Everett turns out to be right there is not a snowball's
> chance in hell of that occurring in any universe.  *
>

Why not?

Einstein (if he had thought a little harder about the instability of the
universe under GR) could have realized that his theory of GR allows for a
constant factor that predicts an expanding universe. Thus Einstein was just
one small step away from predicting the big bang (from his theory alone)
decades before there was any observational evidence to support the notion
of an expanding universe.

In fact, Bruno Marchal (whose field was biology) thought that his theory of
computationalism could not be true because it was incompatible with physics
as he understood it at that time. It was only later, when he discovered
many-worlds that he found a resolution. This is because, his theory of
computationalism, derived purely under the logic of computation, predicted
an ontology of parallel states, and indeterminacy. Things that were not
compatible with his early view that was informed by classical physics. So
it is entirely possible that had computer science gotten a start 100 years
earlier, there would be some who could have anticipated the notion of a
quantum multiverse (just as Bruno did for himself *before* he was aware of
theories in theoretical physics predicting exactly that).

Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260224/7d5efbc0/attachment-0001.htm>