[ExI] AI Motivation revisited

[Stefano Vaj]

On 1 July 2011 21:00, Eugen Leitl <eugen at leitl.org> wrote:
> On Fri, Jul 01, 2011 at 07:11:33PM +0200, Stefano Vaj wrote:
>
>> As long as you go on, replacing along the way dead humans (and perhaps dead
>> planets, since the time I suspect it would take) what's the problem?
>
> The problem is that it doesn't happen, in practice. The longest
> supercomputer batch jobs I know of are all 2-3 years (and have
> to be re-run in order to validate them, despite all the error
> correction tens of thousands of nodes running that long will
> produce uncatched errors).
>
> A typical run takes days to weeks. Most supercomputer
> installations have a lifetime of much less than a decade
> without upgrades. Business upgrade cycles are 2-3 years,
> as older hardware is not worth the juice to keep running.

No, but we are in agreement upon that. There are computations which
are unpractical ("unusable") on insufficiently powerful and/or adept
hardware. What else is new?

The issue here is whether it makes sense to consider AGI as an
emerging property of powerful systems. Or rather, a kind of program
which requires a quite specialised hardware, or more time to be run
than you and I are ready to accept.

> How does less than a second versus a century sound like?

Like a few orders of magnitudes, indeed? :-)

> It would not just be not very performing. It would never finish.

Yes. Hardware which is dramatically inefficient to run a given program
is switched off before completion. This tells us nothing about magical
thresholds, it simply tells us that already beyond the pole of
usefulness, which BTW is much shorter than the remaining time of
universe, things do not get done, *even though they could*.

> My point was that in theory, there's no difference between theory
> and practice. In practice, there is.

Here, I lose you... :-)

> Of course it's less than ideal, taking a theoretical limit. But in
> practice the hardware is both extremely powerful and extremely efficient,
> given the energy constraints.

Yes, this is also my assumption, and this is why I am not buying into
the runaway AGI scenario as the most likely development.

-- 
Stefano Vaj