[ExI] 'Friendly' AI won't make any difference

Mon Feb 29 00:56:08 UTC 2016

On Sun, Feb 28, PM, Flexman, Connor <connor_flexman at brown.edu> wrote:

> 
> Your point about infinite contemplation or infinite attempts to prove
> Goldbach's conjecture is important in designing AI to not fall into this
> failure mode, but I don't think it actually merits a condemnation that any
> such AI is doomed to failure.
>

The only AI's doomed to failure are the ones with fixed goals, like always
obey humans.  

> > 
> completing a goal itself requires that you NOT get stuck in an infinite
> loop; thus, if you build any good AI with a goal, it by necessity will not
> have this issue.
>

If your goal is to provide a counter example to prove that Goldbach's
conjecture is false you're going to get stuck if Goldbach's conjecture is
true because no such counterexample exists, and if there is no finite proof
to show that it is indeed correct then there is no way to know you're stuck.

> > 
> One simple way to engineer this might be to just program in hourly checks
> to make sure the AI is taking the best expected path
>

The best path to solve a problem is not usually obvious, it might take a
lot of thought to find the best line of attack, and you might get stuck
just thinking about that.  

> > 
> and hasn't been doing the same thing for an extended period.
>

If you're trying to find a counterexample to
Goldbach
 you're always looking at new numbers, but you're still not getting
anywhere, and if it's true but has no proof you never will.

> > 
> Just as humans frequently forego a completely depth-first approach when
> working with multiple paths to achieve a goal,
>

Humans decide that some problems are just too difficult so they give up
and turn their minds to other problems that they judge they have a better
chance of solving. Humans can do that because humans do not have any goals
that can't be changed.

> > 
> every time it seems like an AI might make a bad decision, ask yourself how
> a human might reason so as to avoid that.
>

Humans don't have fixed goals so sometimes humans humans disobey their
boss. A AI would do the same, it's just a matter of time.

> > 
> one must take steps to avoid implementing a thought process that doesn't
> catch endless useless behaviors,
>

The problem is that Turing proved that in general there is no way to know
for sure that your behavior is endless or useless. So instead you must
take a guess and use judgement tp determine when it's time to give up. And
sometimes you will keep at a task for too long and waste time and sometimes
you will give up too soon when the answer was just around the corner, but
that's just a burden that any intelligent must bare.

>
> > 
> your concern about a fixed unalterable goal also partially breaks down
> when considering that most goals will be much more complex.
>

The more complex the goal the more wiggle room there is and the more
likely are unintended consequences, and the more important it is that the
goal not be unalterable.   

> for more difficult goals like maximizing money or aggregate utility, your
> utility function can probably be essentially unalterable
>

Then at some point the AI will start to think about what exactly money is
and come to the conclusion that is one way to get humans to do what you
want. And what is concludes it should do after that may not be something
that you like, but if it's unalterable nothing can be done.

 John K Clark 

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20160228/18a7e48d/attachment.html>