[ExI] Unfrendly AI is a mistaken idea.

[Eugen Leitl]

On Thu, Jun 07, 2007 at 09:58:49AM +1000, Stathis Papaioannou wrote:

>    That's the problem with human experts: their agenda may not

As opposed to the other kind of experts? Can you refer me to few of
these, assuming they're any good?

>    necessarily coincide with your own, although at least if you know what

The smarter the darwinian agent, the sooner the whole system will
progress towards more and more cooperative strategies. Only very
dumb and very smart agents are dangerous.

>    potential conflicts will be, like the expert wanting to overservice or
>    recommend the product he has a financial interest in, you can minimise
>    the negative impact of this on yourself. However, one of the main
>    advantages of expert systems designed from scratch would be that they

We can't make useful expert systems designed from scratch, but for a
very few insular applications, vide supra (idiot savant).

>    have no agendas of their own at all, other than honestly answering the

What's in it for them?

>    question posed to them given the available information. How would such
>    a system acquire the motivation to do anything else?

By not being built in the first place, or being outperformed by darwinian 
agents, resulting in its extinction?

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE