[ExI] What might be enough for a friendly AI?

[Dave Sill]

On Fri, Nov 19, 2010 at 2:07 AM, spike <spike66 at att.net> wrote:
> It isn't that simple Mike.  To use that off switch might be considered
> murder.

It's a power switch, not a detonator. The AGI can be restarted after
the situation is analyzed and the containment is beefed up, if
necessary.

> There may not be unanimous consent to use it.

No, it can't require any bureaucratic approval. It has to be a panic
button that anyone can press. Obviously there will be ramifications if
the button is pressed for no good reason.

> There might be
> emphatic resistance on the part of some team members to using it.

That's why it can't be a group decision.

> It might not be clear who is authorized to use it.

Everyone with physical access to the button is authorized to press it.
Data centers have emergency power off buttons readily available,
well-labeled, and usable by anyone in the data center. The purpose
there is different, obviously, but the same mechanism will work just
as well to disable a potentially dangerous AGI.

> Think it over and come back
> tomorrow with a list of reasons why it really isn't as simple as having a
> big power cutting panic button.

I've thought it over for more than a day, and maybe I'm a naive fool,
but I can't see any. I'm all ears, though.

-Dave