[ExI] Yes, the Singularity is the greatest threat to humanity

[Stefano Vaj]

On 17 January 2011 13:34, Eugen Leitl <eugen at leitl.org> wrote:

> On Mon, Jan 17, 2011 at 12:43:35PM +0100, Stefano Vaj wrote:To be able to
> build friendly you must first be able to define friendly.
> Notice that it's a relative metric, both in regards to the entitity
> and the state at time t.
>
> What is friendly today is not friendly tomorrow. What is friendly to me,
> a god, is not friendly to you, a mere human.
>

Exactly. BTW, from the top of my head I cannot really remember the etimology
of the root freund/friend, but "amicus" in Latin, thus in Italian, Spanish,
French etc. basically means somebody who is a co-fighter against somebody
else (inimicus, enemy).

Accordingly, I suspect that exactly as humans and other biological entities
are friendly to some and unfriendly to others, hypothetical anthropomorfic
AGIs would be just the same, and spread their loyalties accordingly. In
particular, I do not see any specific reason why AGIs should be "specieist"
("AGIs all over the world, unite against bio entities!"), a position
dictated by misunderstood Darwinism which even amongst human is far from
universally shared. In fact, being on a rather different food chain would
appear to make it even less likely than it would be the cased for any
biologically-enhanced posthuman species...

 When you bootstrap de novo by co-evolution in a virtual environment and
> aim for very high fitness target is extremely unlikely to be a good team
> player to us meat puppets.
>

Meat puppets who adopt fire and agriculture are not good team players to
those who do not. What else is new? Uploaded, former meat puppets, or
artificial ones, may perform better, even though it is hard to say what they
can in principle do which be different from "fyborg" systems.

 > This in turns means that AGIs in that sense will be from all practical
> > purposes *uploaded humans*, be they modelled on actual individuals or on
> a
>
> Uploaded humans are only initially friendly, of course.


In what sense are they friendly? To whom? Why should they, any more, any
less than when they were in the flesh?


> We're an endangered species. We will need protection, or we will go
> completely extinct. We're the mountain gorillas of the future.
>

Who is an endangered species? Humans of year 2011? Well, the bad news is
that humans of year 1900 are almost completely extinct by now and that
unless emulations capable of running for an undefinite time are established
soon we have a 100% chance of ending up exactly the same way.

If, OTOH, we feel good enough about humans of 1900 having successors and
reproducing, what is the big deal about "children of the mind" taking over
as every biological generation has always done? Not that I would not save
and put away genetic information, it is always nice to have...

In any event, the species of "men without computers" is endangered by the
species "men with increasingly big computers". Removing the "men" from the
second species, should it be actually easy to do, would not change much to
the equation.

-- 
Stefano Vaj
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20110117/2053e9e7/attachment.html>