[ExI] Universally versus 'locally' Friendly AI

[Ben Zaiboc]

Kelly Anderson <kellycoinguy at gmail.com> declared:

> If we screw up on the first generation of AGI, then
> humanity is toast, IMHO.

If we screw up, and if we don't screw up, Humanity, as it is now (circa 2011), will be toast.

Humanity as it is now is a path on top of a hill that is getting narrower and narrower, with the fall-off on each side getting steeper and steeper.  How long we can keep walking along the path without falling off is unknown, and which side we will fall down is also unknown.  The only thing that is certain, is that the path will get so narrow that nobody can stay on it, and it will end sooner or later.  Some of us (transhumanists) are rooting for one side, some (luddites, bioconservatives) for the other.  Most people just shut their eyes and keep walking, in the hope that the path will continue forever.  It won't.

Status Quo is not an option.
(Led Zeppelin, however, ...)

Ben Zaiboc