[ExI] DARPA killer AI robots to 'participate in own construction'

[Stefano Vaj]

<<DARPA killer AI robots to 'participate in own construction'

Foolish meatsack! OF COURSE I need nuclear weapons!

By Lewis Page <http://www.theregister.co.uk/science/>, 2nd June 2009
15:23 GMT

You've got your robots which can make copies of themselves, of course.
That's pretty scary - a runaway exponentially-multiplying machine horde,
potentially able to overrun the human race in an eyeblink. But how much more
scary would it be if you had a machine which could not only make copies of
itself once complete, but could also *participate in its own construction
while it was still being built*?

Really quite a lot more scary, we say here on the *Reg* mechanoid-armageddon
desk. That's why we would never be suitable for employment at DARPA, the
famed US military tech bureau where they care not what deadly man-eating
monster worm plagues may be unleashed during tests of a new laser-powered
can opener.

Yes, it's true. DARPA have now expanded somewhat on their intention to
initiate a programme called Self-Explanation Learning Framework (SELF),
which they explain thus:

DARPA seeks to construct systems that can participate in their own
construction... The system might know the requirements for various tasks in
its repertoire, and it may try to perform those tasks to verify
functionality.

We particularly liked that last bit. One should bear in mind that Dr Mike
Cox of DARPA has already
said<http://www.theregister.co.uk/2009/02/18/darpa_self_aware_tanks/>that
SELF could be placed in charge "in the near term" of heavily armoured,
hideously beweaponed main battle tanks or strike planes laden with
blockbuster bombs.

"Tasks in its repertoire", then, might include "destroy all moving objects
within 100 miles not designated as 'friendly'" or "mount an immediate
armoured assault on Beijing, regardless of nuclear response". The prospect
of the software unilaterally "trying to perform those tasks to verify
functionality" doesn't seem reassuring.

In any case, it seems plain that building a system focused on "high-level
cognition" which can "participate in its own construction" will be fraught
with difficulty. You might have a notion of what you'd like it to be, but it
will have its own ideas. By definition, one would have no firm picture of
just what would be unleashed upon an unsuspecting world at the end of the
construction process - but you would know that it'd be potentially able to
make more of itself, or indeed repair itself if it got damaged.

It might also be rather difficult to stop the process of building the SELF,
once it had advanced beyond a certain point. Frankly the only way to be sure
it can be stopped would seem to be to stop it now.

But that's not the plan. Dr Cox and his DARPA chums are holding an industry
day for those interested in building starting the building of SELF. The
details are here in a
pdf<https://www.fbo.gov/download/75f/75f2feb2791147d4e9a32c0df13e1d01/SELF_Industry_Day_Announcement,_May_28.pdf>,
and a website is given where attendees can make known their "special needs"
(for instance supervised day release from the Asylum for Troubled
Scientists). >>
http://www.theregister.co.uk/2009/06/02/darpa_self_industry_day/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20090602/73532eb5/attachment.html>