[ExI] Unfrendly AI is a mistaken idea.

[Stathis Papaioannou]

On 06/06/07, Eugen Leitl <eugen at leitl.org> wrote:

> >    own. Essentially this is what you are doing when you consult a human
> >    expert, so why would you expect any less from a machine?
>
> When I consult a human expert, I expect him to maximize his revenue
> long-term, and him knowing that I know that.


That's the problem with human experts: their agenda may not necessarily
coincide with your own, although at least if you know what potential
conflicts will be, like the expert wanting to overservice or recommend the
product he has a financial interest in, you can minimise the negative impact
of this on yourself. However, one of the main advantages of expert systems
designed from scratch would be that they have no agendas of their own at
all, other than honestly answering the question posed to them given the
available information. How would such a system acquire the motivation to do
anything else?


-- 
Stathis Papaioannou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20070607/4602521a/attachment.html>