[ExI] Why do the language model and the vision model align?

[Jason Resch]

On Tue, Feb 10, 2026, 5:58 AM Ben Zaiboc via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

> I don't know about anybody else, but I see a distinct difference between
> the reality of the world, that we can't directly perceive but have to
> interpret (or at least invent (or model as best we can)) from our sensory
> input, and Plato's concept of the real existence of abstract forms.
>
> If we take, for an example, Beauty (or Justice, or Homesickness, etc.,
> etc.), Plato's philosophy regards it as a real thing, that exists
> somewhere, somehow, independently of human minds. My philosophy, and I
> suspect (or at least hope) that of most sensible people, holds that it is
> not. That it is completely inside, and dependent on, the human mind, that
> it is actually a product of the human mind and not something belonging to
> 'the real world', independent of any minds.
>
> There are other things that, as far as we know, must be 'real', or at
> least derived from the 'real world' that we can't directly perceive, like
> for instance this keyboard on which I'm typing. To a blind person it's a
> different experience, but still it's the same thing. This is like the
> 'shadows on a cave wall' example, but it's not Platonism.
>
> Another category contains things like the number 4. That's an abstract
> concept, but it could be argued that it represents something that 'really'
> exists, as in, it can be said to be an objective property of certain
> collections of things.
>
> It's hardly surprising that thinking systems would converge on efficient
> ways of representing what exists in the real world, including objective
> properties. That's not the same thing as Platonism, which claims that
> subjective concepts have an existence outside the mind. Which is, of
> course, absurd, or they wouldn't be subjective.
>
> Colour is a popular topic among certain members of this list. Does Colour
> 'really' exist, in a Platonic sense? Having studied how our nervous systems
> process visual information, including how it deals with different
> wavelengths of light, I say no, it doesn't exist as an objective thing
> independent of human (or any) minds. Colour is related to, but different
> from, wavelengths of light. It's subjective, created by our minds from
> information that includes wavelength as well as other things, like
> intensity and contrast, and various properties of our bodies. If you're
> doubtful about this, look up "Magenta". You may be surprised to find that
> it doesn't 'really' exist.
>

Yes, such colors are called "extraspectral colors" they include colors like
beige, brown, white, gray, pink, etc. Most of the colors we can see are not
found as pure colors/frequencies on the spectrum, but are extraspectral,
like magenta.


> It might be interesting to find out if different AI systems create
> representations of colour like we do (do they see magenta?), or if they use
> representations of wavelength instead. I'd guess that if they do create
> internal colours, they'd be subject to the same, or similar, illusions that
> humans are, and different sensory apparatus would lead to different
> representations.
>

An important difference bet how computers represent color and how our
brains interpret it, is that before the data gets to the visual cortex,
color information from the retina is out through the opponent process (this
probably helps us adapt to large sudden changes in brightness).

So instead of receiving raw red, green, blue (RGB) information, our brains
receive: "the ratio of red to green light", "the ratio of blue to
(red+green light)", and "the total amount of red+green+blue light".

Note: red+green gives yellow, and red+green+blue gives white. So we can
rewrite the above as:
"The ratio of red to green light"
"The ratio of blue to yellow light"
"The ratio of white to black"

The opponent process explains several curious facts of our color
experience. Note for example, that because our brains only learn the
relative ratio (e.g. 'am in getting more red light, or green light) this
means the brain can't be aware of red and green at the same time and
location in the visual field, nor can it be aware of yellow and blue at the
same time and location in the visual field.

We can see:
- yellowish reds (oranges)
- yellowish greens (limes)
- bluish greens (teals, turquoises)
- reddish blues (purple, magenta)
- whitish reds (pinks)
- whitish blues (sky blues)
- whitish greens (light greens)

But cannot see these "impossible colors":
- greenish reds
- yellowish blues

Instead we see an entirely different color when these mix, colors that look
nothing like either of the component colors. Yellow looks nothing like
green or red. White looks nothing like blue or yellow.

So if computers process raw RGB data, and if neural networks contain no
higher level process replicating our opponent process, the types and
combinations of colors they might see and experience could be quite alien
from the colors we experience. However, it would not surprise me if neural
networks trained on human color names and relations have convergently
developed an internal opponent process, if only to help it better recognize
human descriptions, names, and relation of colors, and the structure of the
color wheel.


But if they do, that's not evidence in support of Platonism.
>
> In fact, I suspect that Magenta is one of the things that Stefano is
> referring to when he says that "Platonic Ideas have been extensively
> demonstrated to be false by science".
>
> Plato's allegory of the Cave doesn't support Platonism, as it can only
> deal with things that do actually exist outside the cave. Compassion,
> frustration and hunger don't cast shadows on the cave wall, they exist only
> in the minds of the observers.
>
> Jason Resch wrote:
> > I think you may be reading too much into the name "Platonic
> representation hypothesis". The fact that the word "Platonic" is used is
> the name of this hypothesis is not meant as an endorsement or claim to the
> the truth of Platonism.
>
> Ah, ok. Then the name is (very) misleading.
>

I agree. The paper is only talking about concepts derived from data
recorded from the real world. It is not postulating any influence from a
platonic realm.

Instead they are using the word platonic (I think) to refer loosely to his
notion of ideals, but without endorsement of Plato's ontological claims.


> This is just like using the term "substrate-independent" to refer to what
> should be called "substrate-indifferent".


I really like this term "substrate indifferent." Did you originate it? I
may borrow it and want to give proper credit.


It sounds like it refers to the old SF concept of minds that are
> discarnate, not tied to matter at all, but in fact it doesn't. It's
> actually just the opposite of what we tend to call 'carbon chauvinism' -
> the idea that only biological beings can host minds.
>
> We are using terms that are not fit-for-purpose. It's as if someone
> started calling their Thorium Fission power plant a "Cold Fusion
> generator", and everyone goes "yeah, ok, we'll call it that".
>
> It surprises me that people who are so intelligent and imaginative are so
> willing to tolerate and use and even promote, such imprecise and misleading
> language, especially when we all know that miscommunication causes a large
> percentage of the problems in the world.
>
> Anyway, that's my rant over.
>
> So, can I take it that the "Platonic representation hypothesis" in fact
> has nothing to do with Platonism? (The idea that abstract objects are
> asserted to exist in a third realm distinct from both the sensible*
> external world and from the internal world of consciousness).
>

I suppose this problem is natural when dealing with the names of people who
are so prodigious that their names are famous for introducing multiple
concepts. I don't know the best way to avoid it without sacrificing what
may be due credit.

Aristotlean, Newtonian, and Einsteinian, are similarly overloaded terms,
for example.

If I use the term "Newtonian fluid" someone unfamiliar with the term might
think of Newtonian classical physics (in contrast to quantum physics, or
relativistic physics). Someone under that assumption might then think that
a "Non-Newtonian fluid" must involve relativity or quantum mechanics, but
they would be mistaken.



> Are we all in agreement on this?
>


I agree that "Platonic representation hypothesis" has nothing to do with
"the idea that abstract objects exist in a third realm distinct from both
the external world and from the internal world of consciousness."

But I also sympathize with their (admittedly ambiguous and confusing)
naming.

Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260210/6abaaebc/attachment-0001.htm>