[ExI] Unfrendly AI is a mistaken idea.

[Eugen Leitl]

On Wed, Jun 06, 2007 at 08:47:40PM +1000, Stathis Papaioannou wrote:

>    People will want systems that advise them and have no agenda of their

That's what people, as individuals want. But that's not what they're
going to get. Collectively, the system looks for human substitutes
in the marketplace, which, of course, results in complete transformation
once the tools are persons.

>    own. Essentially this is what you are doing when you consult a human
>    expert, so why would you expect any less from a machine?

When I consult a human expert, I expect him to maximize his revenue
long-term, and him knowing that I know that.

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE