[ExI] What might be enough for a friendly AI?

Mike Dougherty msd001 at gmail.com
Fri Nov 19 05:25:13 UTC 2010


On Thu, Nov 18, 2010 at 12:52 PM, Dave Sill <sparge at gmail.com> wrote:
> On Thu, Nov 18, 2010 at 12:12 PM, spike <spike66 at att.net> wrote:
>>
>> That's it Stefano, you're going on the dangerous-AGI-team-member list.  It
>> already has Florent, Samantha, me, now you, and plenty of others are in the
>> suspicious column.  We must be watched constantly that we don't release the
>> AGI, should the team be successful in creating one.
>
> Everyone has to be on that watchlist. You can't assume that anyone is safe.
>

I'd like to preemptively put the AGI itself on the list.

Maybe we could build two of them and make them watchdog each other.
Obviously they'll have incentive to outwit the other - competition in
that sense would be good for them ("build's character")  - when either
one of them agree to let the other out, we turn them both off.
(colluding monsters)

Now we just have to remember to build the off switch first and put it
in a place that we can use easily.  If we depend on the AGI to route
messages to the switch we don't really have a functional killswitch do
we?  I bet they'd figure that out pretty early in the process of
outsmarting us and be all kinds of helpful until we become fat, dumb
and happily dependent on them.

"To Serve Man" is a cookbook!




More information about the extropy-chat mailing list