[ExI] Unfrendly AI is a mistaken idea.

[Stathis Papaioannou]

On 12/06/07, Eugen Leitl <eugen at leitl.org> wrote:

>    So you would give a computer program control of a gun, knowing that it
> >    might shoot you on the basis of some unpredictable outcome of the
> >    program?
>
> Of course you know that there are a number of systems like that, and
> their large-scale deployment is imminent. People don't scale, and
> they certainly can't react quickly enough, so the logic of it
> is straightforward.
>

No system is completely predictable. You might press the brake pedal in your
car and the accelerator might deploy instead, most likely due to your error
but not inconceivably due to mechanical failure. If you were to replace this
manual system in a car for an automatic one, you would want to make sure
that the new system is at least as reliable, and there would be extensive
testing before it is released on the market. Why would anyone forego such
caution for something far, far more dangerous than car braking?



-- 
Stathis Papaioannou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20070612/037de591/attachment.html>