<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><BR><DIV><DIV>On May 28, 2007, at 7:55 PM, Stathis Papaioannou wrote:</DIV><BR class="Apple-interchange-newline"><BLOCKQUOTE type="cite"><BR><BR><DIV><SPAN class="gmail_quote">On 29/05/07, <B class="gmail_sendername">Lee Corbin</B> <<A href="mailto:lcorbin@rawbw.com">lcorbin@rawbw.com</A>> wrote:<BR><BR></SPAN><BLOCKQUOTE class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Suppose that you were a dog, and your intelligence was raised<BR>sufficiently for you to truly understand that your limitless love<BR>for, and obedience to, and worship of your master was the<BR>the consequence breeding? Can't you see that it would change <BR>nothing for you? The movie "AI" did make this point quite well.<BR></BLOCKQUOTE></DIV><BR>This is a good point. We have inherited our top level goal, "survive", from the very first living organism, which would have had no understanding of its meaning. The fact that we now understand what survival means, and that we have been programmed this way by evolution without having any choice in the matter, does not mean that we are inclined to overthrow this goal, even if we could do so. It would be the same if we had been born with the top level goal, "love and obey your master". No matter how well we understood it, how smart we became, we would be no more likely to try to overthrow it than we would be likely to overthrow our will to survive. This does not mean that the top level goal could nevver be overthrown, because people do go mad and kill themselves, but it wouldn't be *as a result of* increased intelligence and understanding. <BR clear="all"><BR></BLOCKQUOTE><BR></DIV><DIV>This line of reasoning has considerable dark side potential. We can and do go beyond our EP in at least some ways. If we truly could not do so then we would always be untrustworthy cosmic rednecks no matter how augmented we someday become. We would also find it impossible to overcome our EP even if it was a matter of our very survival which I think in some ways it is.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>- samantha</DIV><DIV><BR class="khtml-block-placeholder"></DIV><BR></BODY></HTML>