[ExI] Why do the language model and the vision model align?

[Jason Resch]

On Thu, Feb 19, 2026, 3:59 PM Ben Zaiboc via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

> Apologies for not changing the subject line (again!), as this is now so
> far off the original topic. Probably not worth it now, though, as I think
> this has run its course.
>
> On 19/02/2026 18:12, Jason Resch wrote:
> > On Thu, Feb 19, 2026, 7:05 AM Ben Zaiboc via extropy-chat <
> extropy-chat at lists.extropy.org> wrote:
> >
> >     On 17/02/2026 12:05, Jason Resch wrote:
> >
> >     >
> >
> >     >
> >
> >     > On Mon, Feb 16, 2026, 1:33 PM Ben Zaiboc via extropy-chat <
> extropy-chat at lists.extropy.org> wrote:
> >
> >     >
> >
> >     >     On 16/02/2026 16:34, Jason Resch wrote:
>
> > ...
>
> >
> >      Maths is an expression of the properties of the universe, a
> consequence of the particular laws this universe uses, or at least a
> consequence of the way we see them.
> >
> >
> > That is one way to look at it, among many.
> > For example see this section:
> >
> > https://alwaysasking.com/why-does-anything-exist/#Math_Matter_Mind
> >
> > Which shows even among three physicists, they each hold different
> positions.
> >
> > For example, you say that "Math is an expression of properties of the
> universe." But I think it is just as possible that "The universe is an
> expression of properties of mathematics."
> >
> >
> >     We don't have to abandon maths just because it might not be 'true',
> or might not actually exist in some hypothetical mystical plane of
> existence.
> >
> >
> > Those who subscribe to mathematical realism hold mathematical objects to
> exist as concretely as any existing physical universe does. There's nothing
> mystical about it.
>
> If someone tells me that the square root of -1 exists as concretely as the
> monitor sitting in front of me, I'm going to call that mysticism. I don't
> know what else to call it.
>

Imaginary numbers are a crucial element of quantum mechanics. Your
monitor's LEDs are based on quantum mechanical principles. Before your
retina can register a photon emitted from your monitor, quantum mechanics,
and it's imaginary numbers, have already been invoked.



> >
> >      We use maths because it's useful, not because it's true, or
> actually exists.
> >
> >
> > But note that to be useful, a mathematical theory must accurately
> differentiate true from false. So when one of our useful mathematical
> theories says it is true that "$1000 - $995 = $5" also tells us that 9 is
> non-prime because an integer factor of 9 (besides 1 and 9) exists, are we
> not right to say "3 exists"?
> You've lost me there.
>

For the number 9 to be composite (non-prime), the number 3 must exist. So
this is an example where truth and existence are intertwined and
inseparable: the mathematical truth that "9 isn't prime" implies the
existence of a mathematical object (the number 3).

This is my counterpoint to your claim that math only needs to be useful,
and doesn't concern itself with what's true or what exists.


> > What about when the theory says there are primes so large we will never
> be able to compute them? This is an inevitable conclusion if we take our
> mathematical theories seriously.
>
> Well that's simple enough. The theory says that these prime numbers will
> never exist. If there can be such a theory.
>

This is counter to Euclid's theorem (
https://en.wikipedia.org/wiki/Euclid%27s_theorem ) which proved there exist
infinite primes.

So our choice is we either abandon everything fundamental to our
mathematical theories and try to build some patchwork around the
ultrafinitist position you advocate for, or we make peace with and accept
the simpler theory, which says there is no largest integer.


>
> > It is no different from the physicists who takes general relativity
> serious and who concludes, based on the measured curvature of the universe,
> that there exist regions space far beyond the cosmological horizon. They
> are so far away that we will never be able to see them. But these regions
> must exist if our theory of GR is true.
> So these regions of space /have/ been calculated. Which is a different
> thing.
> >
> > In both cases, we are taking established useful theories at their word,
> and using them to predict the existence of things we may never see.
> I think you are confusing things which can be shown to exist and things
> which can't.
>

I thought I was clear that we couldn't see those other positions in space.
However, we can infer these things exist indirectly, through our
observation of things that confirm GR as a theory.

We can show that there are regions of space beyond what we can see, but we
> can't show that there are (or are not) infinitely many primes.
>

In both cases, we are simply relying on the assumption that our given
theory is true.

If you want to abandon the use of theories for making claims of what does
or doesn't exist, then I am afraid you must retreat to solipsism and
abandon the belief in anything existing aside from your current
instantaneous moment of conscious experience. Everything else we believe
exists (the outside world, other people, other minds, the past and future)
is based on theories we assume but can never prove.



>
> >
> >     Do those certain things exist independently of being worked out?
> That's kind of a non-question. Does a falling tree make a noise if nobody
> hears it?
> >
> >
> > Did this physical universe not exist before life arose in it?
> Yes, it did. We can figure that out.
>
> >
> > Would it not then still exist even if no life ever evolved in it?
> That's not a question that can be answered.
>
>
>
But that does seems to follow from your agreement that the universe existed
before there were conscious observers in it.

>
> > ...
> >
> >
> >
> >     > You believe there are more than 52 Mersenne primes, don't you?
> >
> >
> >
> >     I have no beliefs concerning Mersenne primes, mainly because I don't
> understand what they are. I did look up the definition, but that didn't
> help.
> >
> >
> > A Mersenne prime is any prime number that's one less than a power of 2.
> In other words, a prime that when expressed in base 2, consists of all 1s.
> For example:
> >
> > 3: 11
> > 7: 111
> > 31: 11111
> >
> > As of today, only 52 Mersenne primes are known. But it is believed more
> (and possibly infinitely many) exist. Let's assume there are more.
> >
> > Then consider the following statement:
> >
> > "A 53rd Mersenne prime exists."
> >
> > Is such a statement true?
> You just said 'it is believed...', so the answer to your question is "some
> people believe so".
> >
>

Yes, I said let's assume that there are more left to be found, so we can
focus on what we each mean when we say something exists (or doesn't) before
any person sees it.  Or if it helps, imagine anyone when only 51 Mersenne
primes were identified. I would say the 52nd Mersenne prime exists (despite
the fact that no one had yet identified it at that time). Would you say
that it did not exist until the point in time some human found it?

If you say no, then would your answer change if some alien race on the
other side of the galaxy had found it already? If this changes things, that
seems to make math into a very subjective thing, whose theorems and truths
could vary between each person. I don't know how to make sense of such a
view of mathematics.


> Or does it only become true after someone finds it?
> After someone finds it, it is certainly true. As to  whether it's true
> before then, well, "some people believe so" is the most you can say.
> > ...
> >
> >
> >      I don't think that someone in ancient mesopotamia who said "there
> are no buildings half-a-mile tall" could reasonably be said to have been
> wrong, despite the fact that, given the right circumstances, it's possible
> to create buildings half-a-mile tall.
>

If he said such buildings were not possible he would be wrong. If he said
he was unaware of any such buildings he would be right. I don't see how
these statements are meant to show that objective facts change with
discovery, the qualifiers of "possible" "exists" and "is known to me" are
each very different.

As I see things, there is no mathematical truth that is true for one
person, in one time, or in some place, that is false for some other person,
in a different time, or in a different place.

We can't even say this much about physical facts, since some believe
constants of physics can change over time, or they can be different in
different universes. So in this sense, physics is a less objective field
than mathematics.


>
> >
> >
> > I am not talking about possibilities which may or may not exist, but
> rather, conclusions we must accept if the theories we use and rely on
> happen to reflect the underlying reality.
> Which applies to tall buildings as much as to mathematics. We know for a
> fact that half-mile-tall buildings can exist. Nevertheless, we don't
> conclude that the mesopotamian was wrong.
>

I hope my point above clarified any confusion about this.


>
> >
> >     > There are different forms of existence.
> >
> >     >
> >
> >     > There is existence defined by being mutually causally interactive
> (what we normally think of as physical existence, or existing within this
> universe).
> >
> >     >
> >
> >     > But then there is also existence for things which are acausal. For
> example, two bubble universes in eternal inflation that will never
> interact, or two decohered branches in many worlds, or even just other
> universes with different laws, which we presume must exist to explain the
> fine tuning of the laws of our own universe. In what sense do these other
> universes exist?
> In the 'hypothetical' sense, unless they are proven to be factual.
>

Nothing can be proven. Even 2+2 = 4 cannot be proven, because any proof of
such must make assumptions about the axioms, which themselves can never be
proven (without making still further unprovable assumptions).

So we are stuck with either falling into solipsism of the worst kind, or
living with the understanding that all our beliefs concerning what exists
beyond our immediate consciousness is based on theory and assumptions, for
which we can have varying degrees of confidence.


>
> >
> >     >
> >
> >     > Are they still worth of the full "concrete physical existence"
> when we can't see them and can't interact with them? Or should their
> existence be demoted to inferred/abstract/theoretical?
> >
> >     >
> >
> >     > If the latter, isn't that the same sort of existence that
> mathematical object have? Other physical universes can be studied via
> simulation, we can analyze their properties, what structures exist as a
> result of their different laws, etc.
> >
> >     >
> >
> >     > The abstract sort of existence that other possible universes have
> seems to be, to be the same sort mathematical objects have.
> >
> >
> >
> >     Ok, we can agree on that.
> >
> >     It could be called 'imaginary, but with rules'. Which is a subclass
> of 'imaginary'.
> >
> >
> > I think other physical universes deserve a category slightly higher than
> imaginary.
> Don't think I mean anything derogatory by the word 'Imaginary'. I have a
> lot of respect for the imaginary. It could even be said to be the key
> factor that makes us human. And allows us to devise things like maths and
> science.
>
> We don't know that there are other physical universes, we only actually
> know about this one. We can theorise about them, though ('imaginary with
> rules').
>

The experiences we point to that justify our belief in the universe we
think we see, are the same sorts of experiences we point to to justify our
belief in the universes we don't see, or the parts of this universe we
don't or can't see.

In the end, there is only immediate conscious experience, and our
inferences from those to some greater reality.


> >
> > Certainly the physicists who postulate the actual existence of other
> universes (to explain cosmological fine tuning observed in our universe)
> are doing something a little more serious than contemplating things in the
> same category as Santa.
> Absolutely, hence my simplified classification system.
> >
> > Likewise, I think mathematicians who devote their lives to thinking
> about objects in math are doing more than playing imaginary games.
> That is exactly what they're doing. We have countless examples showing
> this, from Einstein through William Hamilton to Kekule (yes, chemistry
> rather than maths, but it's the same process: playing imaginary games (with
> the relevant rules)).
>


For me the word imaginary doesn't work because it conveys a sense of
arbitrariness and subjectivity that isn't there. Newtown wasn't free to
make up any laws he wanted, reality led him to "F = ma". Likewise
mathematicians aren't free to make up whatever axioms they like. They, like
Newton, are constrained by reality.

>
>
> > Quite often in history, mathematicians had already laid the groundwork
> for physical theories not yet conceived.
> >
> >
> >
> >
> >     So I'd propose a simple classification system:
> >
> >
> >
> >     A) Stuff that physically exists (ducks, people, sofas, stars,
> magnetic fields, etc.)
> >
> >
> >
> >     B) Things that are imaginary (exist as information patterns in
> minds: Santa, Jealousy, Immoveable objects, Other minds, etc.)
> >
> >         B1)  Imaginary things that conform to specific rules (Maths,
> Cricket scores, Cutlery etiquette, etc.)
> >
> >
> > A gold start, but I don't think there is a clear spot for:
> >
> > - Regions of space so far away we can't see them or interact with them?
> > - Other branches of the wave function?
> > - Actually existing alternate universes with different laws?
> >
> > Are all these things physical?
> Ok, to expand it:
>
> A) Physical stuff
>     A1) Stuff that can be demonstrated to physically exist (ducks, people,
> sofas, stars, magnetic fields, etc.)
>     A2) Stuff that can be shown by theory to physically exist (space-time
> beyond our light-cone, black holes, quarks, etc.)
>
> These two categories are closely related, with experiments sometimes
> proving theories, and sometimes giving birth to new ones
>

What is the significant difference between accepting the existence of
something because a physical theory suggests it, can accepting the
existence of something because a mathematical theory suggests it? Note that
in both cases, we are using empirically derived theories to make inferences
about the content of external reality.



> B) Things that are imaginary (exist as information patterns in minds:
> Santa, Jealousy, Immoveable objects, Other minds, etc.)
>     B1) Imaginary things that conform to specific rules (Maths, Cricket
> scores, Cutlery etiquette, etc.)
>
> I don't know how to categorise 'other branches of the wave function'
> because I don't exactly know what it means. Sounds like quantum stuff,
> though, and might mean the same thing as 'other universes'?
>

You can treat them as functionally other universes, which have theoretical
from quantum mechanics. They are other universes that can, in certain
circumstances, interact with our own, until they deochere, in which case
further interaction is prevented.


> >
> > If so, consider that string theory suggests there are at least 10^500
> different sets of physical laws. All these different universes existing as
> different laws as a result of one mathematical foundation of string theory
> equations.
> >
> > According to strong theory, all these universes physically exist.
> According to religious theory, so does the holy ghost.
>

Just before you said we can use physical theories to justify beliefs in
physical things we can't see. Are you now denying this?


>
> > But what makes the equations of string theory special? Why shouldn't
> there be universes that follow other equations besides those of strings? If
> other equations defining other universes, are no less valid than string
> theory, then the line between physical existence and mathematical existence
> dissolves.
> >
> > Physical existence is nothing more than mathematical existence. And we
> are back to Platonism. Or as Tegmark describes it, the mathematical
> universe hypothesis:
> >
> > https://en.wikipedia.org/wiki/Mathematical_universe_hypothesis
>
> Well, I think it's the other way around.
>
> In the end, as John Clark is fond of reminding us, there's no disputing
> matters of taste.
>

It would be a matter of taste except for the fact that there is strong
empirical evidence that it is as I say, that the physical emerges from the
mathematical/computational. Until and unless you can show some alternate
explanation or theory for these observations, this is simplest (and so far
the only known) answer for these facts:

The logician and computer scientist Bruno Marchal showed that
computationalism and arithmetical realism predict a physics with quantum
logic, quantum indeterminacy, quantum non-locality, and an ontology of
parallel states (Marchal, 2001).
https://iridia.ulb.ac.be/~marchal/publications/CC&Q.pdf

The computer scientist Russell Standish assumed observation within an
infinite plenitude and showed he could derive the linearity of physical
law, Occam's razor, and the Schrödinger equation (Standish, 2006).
https://www.hpcoders.com.au/theory-of-nothing.pdf

The quantum physicist Markus Müller detailed how algorithmic information
theory predicts that most observers will find themselves in universes with
time, a beginning, and governed by simple, computable, probabilistic laws
(Müller, 2020).
https://quantum-journal.org/papers/q-2020-07-20-301/

The computer scientist and physicist Stephen Wolfram details how all
computations playing out in all possible ways explain why observers will
see a universe with the second law of thermodynamics, general relativity,
and quantum mechanics (Wolfram, 2021b).
https://writings.stephenwolfram.com/2021/03/what-is-consciousness-some-new-perspectives-from-our-physics-project./

Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260219/e2617f19/attachment.htm>