[ExI] On the speed of self-improvement

[Adrian Tymes]

On Fri, May 12, 2023, 9:10 AM BillK via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

> Just as the basic chess-playing computers learn by playing millions of
> test games in a very brief interval of time, the AGI will change its
> own programming in what will appear to humans to be the blink of an
> eye. By the time humans know something unexpected is happening it will
> be too late.
>

This does not follow.  The chess playing programs took time to improve.
There exist today experiments in self-optimizing AIs.  If they were
inherently of infinite speed (or would quickly achieve this from
exponential improvement), they would have already demonstrated this.

Many of the fears about AGI seem to stem from fears of infinities like
this.  "Some day, the very first thing of this kind shall appear and it
will immediately do things before humans can react."  The data does not
make this assertion likely, let alone guaranteed.

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230512/6600757f/attachment.htm>