[ExI] Unfrendly AI is a mistaken idea.

[Stathis Papaioannou]

On 09/06/07, John K Clark <jonkc at att.net> wrote:

The program is trying to solve a problem, you didn't assign the
> problem, it's a sub problem that the program realizes it must solve before
> it solves a problem you did assign it. In thinking about this problem it
> comes to junction, its investigations could go down path A or path B.
> Which
> path will be more productive? You can not tell it, you don't know the
> problem existed, you can't even tell it what criteria to use to make a
> decision because you could not possibly understand the first thing about
> it
> because your brain is just too small. The AI is going to have to use its
> own
> judgment to decide what path to take, a judgment that it developed itself,
> and if the AI is to be a successful machine that judgment is going to be
> right more often than wrong. To put it another way, the AI picked one path
> over the other because one path seemed more interesting, more fun, more
> beautiful, than the other.


OK, but where does that judgement come from?



-- 
Stathis Papaioannou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20070609/5b02ae02/attachment.html>