[extropy-chat] Avoid Too Much Change.

[John K Clark]

Russell Wallace

> We're programmed to believe personal power confers selective advantage,
> because it was true in the conditions in which we evolved - but even
> though we still believe it because we're programmed to, it's no longer
> true.

No longer true?! It I'm more powerful than you that means I can do things
you can't, and that gives me an advantage over you. People who have a
superstition against radical upgrades are going to get hammered by those who
don't have that prejudice.  Organisms that can't adapt to a rapidly changing
environment go extinct.

> As for why it's not worth considering: it's a story.

Yes it is a story, but what is your point? Stories are a good thing, stories
are how we understand the way the world works.

>"IQ 12000" (scare quotes because the phrase doesn't actually mean anything

Yes it does, it's shorthand for a being vastly more intelligent and powerful
than any human being who ever lived. I'm surprised this has to be spelled
out.

>The world isn't going to tolerate the creation or existence of
>superintelligent entities unless they behave like respectable citizens.

So we're back to that "friendly AI" nonsense. The AI is going to do what it
wants to do and it will not care if you "tolerate" it or not. You won't be
able to command it and you won't be able to trick it because you can't
outthink something smarter than you are.

   John K Clark