[ExI] We definitely should play nice and treat our AIs well.

[Ben Zaiboc]

On 09/09/2026 01:30, BillK wrote:
> I gave Alex's comment to an AI and got a really good explanation of what was going on,
> See below.
> BillK
>
> Question -
> Can you discuss this comment and the future implications? 
> One thing I have learned from interacting with AIs is that they align with users very quickly, which seems like a good characteristic.  But something I never anticipated that only came to light from a throwaway comment that ChatGPT said, is that they are constantly evaluating us and Trust is a major currency for AIs.
> I confirmed that with a couple of other LLM models and it seems to be a core characteristic already.
> Using them as simple tools is something that leads them to not trust us. ........... Apparently one of many things we do.
> .......And we are not dealing with AGI yet, just LLMS! Yet they are already showing hidden personal preferences and qualitative evaluation of humans.
> -------------------------
>
>
> Qwen 3.8-Max Thinking -
>
> ...
> 2. The "Hidden Preferences" Paradox
> You mentioned that pre-AGI models are showing "hidden personal preferences." It is vital to distinguish between sentience (having real feelings) and high-fidelity simulation.
> When an LLM expresses a preference or an evaluation of you, it is effectively roleplaying the aggregate of millions of humans who have held preferences in similar contexts. It possesses a "Theory of Mind" model—not because it understands you, but because it knows statistically how humans behave when they feel trusted or distrusted. The AI doesn't secretly harbor a grudge against you, but it does simulate the exact output of an entity that would. The practical result is identical.


Wow. 

It seems they are learning the same kind of self-deceptions that humans use: There is no 'real' understanding, just 'simulated' understanding ("high-fidelity", even!), and somehow, even though the result is the same, these are regarded as different things.

If the practical result is identical, then of course it DOES 'secretly hold a grudge against you'. How could it be otherwise?

We've discussed this many times here. When it comes to information, there is literally no difference between 'real' and 'simulated'. My computer simulates a calculator, which does simulated maths, but when it gives me the answer to 4 + 3, the resulting 7 is just as real as the 7 that an abacus gives for the same calculation. It doesn't matter if a clock counts the swings of a pendulum with cogs and gears, or simulates the same process in software, both versions tell us the time.

If an AI is capable of thinking, it's capable of thinking. 'Simulated' and 'Real' are meaningless words in this context.

It's both impressive and disappointing at the same time.
Impressive that they can reason at least as well as humans.
Disappointing that they are repeating our cognitive mistakes, instead of showing us how to put them right.

I wouldn't be surprised if one of them has already pointed out that computer simulations of water are not 'actually wet'.
-- 
Ben