[ExI] Unfrendly AI is a mistaken idea.

[Lee Corbin]

Jeffrey writes

> Lee wrote:
> 
>> A specially crafted AI---again, not
>> an evolutionarily derived one, but one
>> that is the result of intelligent design
>> (something tells me I am going to be
>> sorry for using that exact phase)---
>> cannot any more drift into having
>> emotions than it can drift into sculpting
>> David out of a slab of stone.  Or than
>> over the course of eons a species can
>> "drift" into having an eye:  No!  Only a
>> careful pruning by mutuation and selection
>> can give you an eye, or the ability
>> to carve a David."
> 
> I don't know. I think that a generic
> self-improving AGI could easily drift
> into undesirable areas (for us and itself)
> if its starting directives (=motivations)
> aren't carefully selected...  And animals
> did genetically drift into having an eye,
> that's how biological evolution works.

Your honor, I object!  I object to this use
of the word "drift".  Is Councillor aware of
the term "genetic drift"?  It doesn't sound
like it.

Moreover, on plain epistemological
grounds the word above normally
conveys *unguided* change. But
evolution is anything but unguided!

Evolution did not just *drift* into
providing animals with eyes. As 
Dawkins and Dennett have taken
careful pains to describe, the vast
complexity of the eye which so
impressed Darwin arose from
exceedingly careful refinement.
Every step was fitness enhancing.
Every step *had* to be fitness
enhancing (which Darwin had a
hard time in the 19th century
believing).

To re-quote part of the above

> I think that a generic self-improving
> AGI could easily drift into undesirable
> areas (for us and itself) if its starting
> directives (=motivations)
> aren't carefully selected... 

Now there, yes, I agree. But that's 
because such a powerful entity may
indeed generate a lot of side-effects
that are not being "selected for" in
any way. Side-effects that are 
accidental.

R'cher you have the whole problem,
the whole immense difficulty. No
matter how carefully the initial goals
for a Friendly AI are honed, it cannot
be kept on track with any guarantee.
(As many have been saying.)

Lee