<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><BR><DIV><DIV>On May 27, 2007, at 12:59 AM, Samantha Atkins wrote:</DIV><BR class="Apple-interchange-newline"><BLOCKQUOTE type="cite"><BR><DIV><DIV>On May 26, 2007, at 2:30 PM, Brent Allsop wrote:</DIV><BLOCKQUOTE type="cite"> My original statement had this:<BR> <BR> Name: <B>Such concern is mistaken</B><BR> One Line: <B>Concern over unfriendly AI is a big mistake.</B><BR> <BR> If you agree, Russell, then I propose we use these for the supper camp that will contain both of our camps and have the first version of the text be something simple like:<BR> <BR> Text: <B>We believe the notion of Friendly or Unfriendly AI to be silly for different reasons described in subordinate camps.</B><BR> <BR></BLOCKQUOTE><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Subgroup: No effective control.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>The issue of whether an AI is friendly or not is, when removed from anthropomorphism of the AI, an issue of whether the AI is strongly harmful to our [true] interests or strongly beneficial. It is a very real and reasonable concern. However, it is exceedingly unlikely for a very advanced non-human intelligence that we can exert much leverage at all over its future decision or the effects of those decisions on us. Hence the issue, while obviously of interest to us, cannot really be resolved. Concern itself however is not a "big mistake". The mistake is believing we can actually appreciably guarantee a "friendly" outcome.</DIV></DIV><BR></BLOCKQUOTE><BR></DIV><DIV>There is also some danger that over concern with insuring "Friendly" AI, when we in fact cannot do so, slows down or prevents us from achieving strong AI at all. If the great influx of substantially >human intelligence is required for our survival then postponing AI could itself be a real existential risk or failure to avert other existential risks.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>- s</DIV><BR></BODY></HTML>