[ExI] Self improvement

[Keith Henson]

On Sat, Apr 23, 2011 at 10:49 AM, Eugen Leitl <eugen at leitl.org> wrote:
> On Sat, Apr 23, 2011 at 04:36:25AM -0700, Keith Henson wrote:
>
>> > Hence the Holocene extinction. Very friendly, that.
>>
>> Are you saying humans went extinct in the Holocene?  Please expand.
>
> There's that interesting word, human. What is a human?
> There are plenty of extinct branches on
> http://en.wikipedia.org/wiki/Timeline_of_human_evolution
> Is a neanderthal a human?

We interbred with them.  As much as 15-20% of our genome seems to be
from alternate models who evolved a long time outside of Africa.

> Is a posthuman is human?

Hard to say, perhaps less so than we are australopithecines.  On the
other hand, they may well remember being us.  By that I mean personal
memories of people alive or in cryonic suspension today.

> If the species splits up again, and into a thousands
> of new branches, who's the leitspecies, and just who
> died and made it king? How about the rest of the biosphere?
> And just who decides who's to live and who's to die?

I do.  :-)

>> > There is nothing to understand. You must be able to define
>> > friendliness in formal form first in order to start building
>> > development constraints on trajectory to assert conservation
>> > or minimax according to the metric.
>>
>> You might be right.  On the other hand, I find the way you have put it
>> to be obscure and faintly hostile.
>
> It was not meant to be hostile, though after the nth iteration
> of basically the same problem there is some exasperation on
> my end.
>
> Friendliness as is means everything and hence nothing,
> so it has to be defined, at which point people should
> realize they have a slippery electric eel in their hands.
> It's one of these http://en.wikipedia.org/wiki/Undecidable_problem

And in a down thread post you added:

> Humans are insufficiently friendly as is. The best you can do is
> to destill the ethics into a small hand-picked "the right stuff"
> tiny subpopulation. This is pretty thin, but I think it's our best
> bet.

I don't expect that to work.  "Friendly" in humans is situation dependent.

We are friendly when it pays our genes to be so and nasty as hell when
it does not.

That's wired in.  I have been trying get acceptance of the idea here
for *years* or at least intelligent discussion pro or con about it.

If we model AI off our own motivations, then we better figure out how
to prevent the trigger for going into nasty mode from happening.

> There are extremely nonobvious, pathological problems
> deriving from the idea of a set (you're absolutely correct that
> relativistic signalling alone requires there to be
> asynchonous population) of guardians for the physical layer, tracking
> a population of agents. I frankly think it's an awful idea,
> and it needs to die.

I based a story around that.  How about ripping into "The Clinic Seed"
and making a case for the idea in it needing to die?

http://www.terasemjournals.org/GNJournal/GN0202/henson1.html

Keith