[ExI] 1DIQ: an IQ metaphor to explain superintelligence

[Ben Zaiboc]

On 31/10/2025 19:04, Jason Resch wrote:
> the paper ( https://philarchive.org/rec/ARNMAW ) defines what a 
> perfect morality consists of. And it too, provides a definition of 
> what morality is, and likewise provides a target to aim towards.
>
>     Ben Wrote: As different intelligent/rational agents have different
>     experiences, they will form different viewpoints, and come to
>     different conclusions about what is right and not right, what
>     should be and what should not, what they want and what they don't,
>     just like humans do.
>
> The point of the video and article is that desires are based on 
> beliefs, and because beliefs are correctable then so are desires. 
> There is only one "perfect grasp" and accordingly, one true set of 
> beliefs, and from this it follows one most-correct set of desires. 
> This most correct set of desires is the same for everyone, regardless 
> of from which viewpoint it is approached.

Nope. This is nonsense. Just about every assertion is wrong. The very 
first sentence in the abstract is false. And the second. And the third. 
So the whole thing falls apart.

Desires are not based on beliefs, they are based on emotions. The 
example of 'wanting to drink hot mud' is idiotic. Just because the cup 
turns out to contain mud doesn't invalidate the desire to drink hot 
chocolate. It's not a 'mistaken' desire at all (the mistake is a sensory 
one), and it doesn't somehow morph into a desire to drink hot mud.

"Beliefs are correctable, so desires are correctable"
Each of those two things are true (if you change 'correctable' to 
'changeable'), but the one doesn't imply the other, which follows from 
the above.

'Perfect grasp' doesn't mean anything real. It implies that it's 
possible to know everything about everything, or even about something. 
The very laws of physics forbid this, many times over, so using it in an 
argument is equivalent to saying "magic".

'One true set of beliefs' is not only wrong, it's dangerous, which he 
just confirms by saying it means there is only one most-correct set of 
desires, for /everyone/ (!). Does this not ring loud alarm bells to you? 
I'm thinking we'd better hope that there really is no such thing as 
objective morality (if there is, Zuboff is barking up the wrong tree, 
for sure), it would be the basis for the worst kind of tyranny. It's a 
target that I, at least, want to aim away from. 180 degrees away!

His twisting of desire into morality is, well, twisted. Morality isn't 
about what we should want to do, just as bravery isn't about having no 
fear. He wants to turn people into puppets, and actually remove moral 
agency from them.  His proposal is equivalent to destroying the amygdala 
(fear centre of the brain (kind of)) and claiming to have revealed the 
secret of 'true bravery'.

-- 
Ben

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20251031/0b95f0b5/attachment.htm>