<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>On Nov 28, 2010, at 6:45 PM, Keith Henson wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Verdana; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><blockquote type="cite">If there is no way, even in principle, to algorithmically determine beforehand whether a given program with a given input will halt or not, would an AI risk getting stuck in an infinite loop by</blockquote></span></blockquote><div><span class="Apple-style-span" style="border-collapse: separate; font-family: Verdana; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><blockquote type="cite"><blockquote type="cite">messing with its own programming?</blockquote></blockquote><br></span></div><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Verdana; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; ">Sure there is. Watchdog timers, automatic reboot to a previous version.<br></span></blockquote><br></div><div>Right, but that would not be possible in a intelligence that operated on a strict axiomatic goal based structure, like the one with "obey human beings no matter what" being #1 as the friendly (slave) AI people want. Static goals are not possible because of the infinite loop problem. In Human beings that "watchdog timer" to get you out of infinite loops is called "boredom", sometimes it means you will give up after you seem to have made no progress just before you would have figured out the answer, but that disadvantage is the price you must pay to avoid infinite loops, there just isn't any other viable alternative. </div><div><br></div><div> John K Clark</div><br></body></html>