[extropy-chat] Superintelligence

[ben]

I did say 'incomprehensible'. That means we *don't know* what it's
motivations, it's ideas about what's worthwhile or not, or it's sense of
humour might be. Discussing how the economics of strip-mining a planet
might influence the actions of a powerful AI, makes as much sense as a
beaver being critical of the Sydney Opera House.

Mikhail John wrote:
> I'm assuming that it would be difficult to maintain a distributed 
> intellect while boiling oceans and ripping continents apart

Why on earth would you assume that?

Even i can think of ways to do that, and i'm a very long way from the
kind of intellectual ability that we are assuming an AI would have. But
even if i couldn't, that makes no difference.

My whole point is that we cannot predict the abilities or motivations of
a hugely powerful AI. We'd be wasting our time (we are wasting our time,
sadly) trying.

> Now that I've thought about it, the chance that an AI will leave and 
> ignore us completely ...

You, me, all of us, have absolutely no clue what an AI would be
able/want to do. Not only is it pointless to fool ourselves into
thinking we have got a clue, it's probably dangerous.

ben