[ExI] Hard Takeoff

[Mike Dougherty]

On Thu, Nov 18, 2010 at 12:11 AM, The Avantguardian
<avantguardian2020 at yahoo.com> wrote:
> On the other hand, if the seed AI is able to actually rewrite the code of it's
> intelligence function to non-recursively improve itself, how would it avoid
> falling victim to the halting roblem? If there is no way, even in principle, to
> algorithmically determine beforehand whether a given program with a given input
> will halt or not, would an AI risk getting stuck in an infinite loop by messing
> with its own programming? The halting problem is only defined for Turing
> machines so a quantum computer may overcome it, but I am curious if any SIAI
> people have considered it in their analysis of hard versus soft takeoff.

Perhaps simple boredom is a nice limitation for infinite loops?  Maybe
the AI expresses boredom in terms of ROI and energy efficiency.  What
keeps our attention devoted to certain ideas and prevents our
attention from being devoted to others?