[ExI] uploads again
rafal.smigrodzki at gmail.com
Mon Dec 24 08:45:14 UTC 2012
On Sun, Dec 23, 2012 at 10:47 AM, Anders Sandberg <anders at aleph.se> wrote:
The more agents there are, the more likely at
> least one will be very optimistic, and act - even when the consequences are
> likely bad, and correctly judged by the other agents. So we should expect
> that in the case of unleashing new technologies there will be mistakes done,
> especially if there is a tight race.
### Indeed. I see this as one of the main reasons to rush headlong
into developing Friendly AI: The first successful movers in this field
are likely to at least be smart but once every Tom Dick and Harry can
cobble together a self-enhancing AI from off-the-shelf modules, some
of them will do something stupid, simply because many are stupid, and
some are mean. This consideration is balanced by the likelihood of the
smart folks making mistakes from rushing too fast forward.
I expect a narrow escape for humanity, or perhaps an ignominious end.
More information about the extropy-chat