[ExI] Unfrendly AI is a mistaken idea.

[Eugen Leitl]

On Sun, Jun 17, 2007 at 04:22:04PM +1000, Stathis Papaioannou wrote:
> 
>    The most frightening thing for some people contemplating a
>    technological future is that they will somehow be forced to become
>    cyborgs or whatever lies in store. It is of course very important that

It is a reasonable fear, because this is what they probably must do,
to keep up with the joneses.

We're leaving in slowtime, still expanding, very far from the equilibrium.
This place is not exactly hypercompetitive. Nevertheless, according
to Guns, Germs and Steel there have been several waves of expansion,
and several cultures and technologies becoming dominant in rather
fast waves, including much human death. We're more civilized today,
and prefer memetic warfare and trade, but in places with very high
population density relative to the sustaining capacity of the ecosystem
there are periodical genocide waves happening, too.

>    no-one be forced to do anything they don't want to do. Interestingly,
>    aside from some small communities such as the Amish, the differences
>    between adoption rates of new technology have almost always been to do
>    with differences in access, not a conscious decision to remain
>    old-fashioned. There won't be coercion, but there will be seduction.

If AI turns out to be far more difficult, or safer than we think, then
we do have a lot of time to do what we do now, only more so. It would
be a slow Singularity, with almost everybody making it.