<div class="gmail_quote">On 17 January 2011 13:34, Eugen Leitl <span dir="ltr"><<a href="mailto:eugen@leitl.org">eugen@leitl.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div class="im">On
Mon, Jan 17, 2011 at 12:43:35PM +0100, Stefano Vaj wrote:To be able to
build friendly you must first be able to define friendly.<br></div>
Notice that it's a relative metric, both in regards to the entitity<br>
and the state at time t.<br>
<br>
What is friendly today is not friendly tomorrow. What is friendly to me,<br>
a god, is not friendly to you, a mere human.<br></blockquote><div class="im"><br>Exactly.
BTW, from the top of my head I cannot really remember the etimology of
the root freund/friend, but "amicus" in Latin, thus in Italian, Spanish,
French etc. basically means somebody who is a co-fighter against
somebody else (inimicus, enemy). <br><br>Accordingly, I suspect that
exactly as humans and other biological entities are friendly to some and
unfriendly to others, hypothetical anthropomorfic AGIs would be just
the same, and spread their loyalties accordingly. In particular, I do
not see any specific reason why AGIs should be "specieist" ("AGIs all
over the world, unite against bio entities!"), a position dictated by
misunderstood Darwinism which even amongst human is far from universally
shared. In fact, being on a rather different food chain would appear to
make it even less likely than it would be the cased for any
biologically-enhanced posthuman species... <br><br>
</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
When you bootstrap de novo by co-evolution in a virtual environment and<br>
aim for very high fitness target is extremely unlikely to be a good team<br>
player to us meat puppets.<br></blockquote><div><br>
Meat puppets who adopt fire and agriculture are not good team players to
those who do not. What else is new? Uploaded, former meat puppets, or
artificial ones, may perform better, even though it is hard to say what
they can in principle do which be different from "fyborg" systems.<br>
<br>
</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div class="im">> This in turns means that AGIs in that sense will be from all practical<br>
> purposes *uploaded humans*, be they modelled on actual individuals or on a<br>
<br>
</div>Uploaded humans are only initially friendly, of course. </blockquote><div><br>
In what sense are they friendly? To whom? Why should they, any more, any less than when they were in the flesh?<br>
<br>
</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">We're an endangered species. We will need protection, or we will go<br>
completely extinct. We're the mountain gorillas of the future.<br clear="all">
</blockquote></div><br>
Who is an endangered species? Humans of year 2011? Well, the bad news is
that humans of year 1900 are almost completely extinct by now and that
unless emulations capable of running for an undefinite time are
established soon we have a 100% chance of ending up exactly the same
way.<br>
<br>
If, OTOH, we feel good enough about humans of 1900 having successors and
reproducing, what is the big deal about "children of the mind" taking
over as every biological generation has always done? Not that I would
not save and put away genetic information, it is always nice to have... <br>
<br>
In any event, the species of "men without computers" is endangered by
the species "men with increasingly big computers". Removing the "men"
from the second species, should it be actually easy to do, would not change much to
the equation.<br>
<br>-- <br>Stefano Vaj<br>