<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
On 03/06/2011 05:18 AM, Amon Zero wrote:
<blockquote
cite="mid:AANLkTinEDCQWaFnRqjbfqO20xCCmhRQ+RhJz9KptAMs=@mail.gmail.com"
type="cite"><span class="Apple-style-span" style="font-family:
arial,sans-serif; font-size: 13px; border-collapse: collapse;
color: rgb(51, 51, 51);">
<div><span style="color: rgb(34, 34, 34);">Hi All - </span></div>
<div><span style="color: rgb(34, 34, 34);"><br>
</span></div>
<div><span style="color: rgb(34, 34, 34);">I've been thinking
about AGI and Friendliness. Yes, I know, a minefield to say
the least. Specifically, I've been taking this matter and
comparing it to early Extropian notions about
libertarianism and technological progress, and the
comparison suggests what might be a new question. (Something
that I daresay Ben Goertzel has considered, but I don't have
him to hand, as it were).</span></div>
<div><span style="font-family: arial,sans-serif; font-size:
13px; border-collapse: collapse; color: rgb(34, 34, 34);"><br>
</span></div>
<div><span style="font-family: arial,sans-serif; font-size:
13px; border-collapse: collapse; color: rgb(34, 34, 34);">So,
I remember a piece of Max's (IIRC), in which he made the
case that too many governmental controls on technological
development would only ensure that less-controlled countries
would develop key technologies first. Within reason, that
sounds a plausible claim to me. Universally Friendly AGI, of
the sort that SIAI contemplates, seems to be a textbook case
of constrained technological development. i.e. it seems
reasonable to expect that non-Friendly AGI would be easier
to develop than Friendly AGI (even if FAI is possible, and
there seem to be good reasons to believe that universally
Friendly superhuman AGI would be impossible for humans to
develop).</span></div>
<div><span style="font-family: arial,sans-serif; font-size:
13px; border-collapse: collapse; color: rgb(34, 34, 34);"><br>
</span></div>
</span></blockquote>
<br>
You mean Unfriendly with no real definition of what "Friendly" is?
You mean requiring absolute proof of no harm to proceed? This is
known as the Precautionary Principle and it will most certainly stop
progress dead wherever it is applied. We cannot define in a
provably correct way or enforce in a provably foolproof way
"Friendliness" to humans much less universally (whatever that
means).<br>
<br>
<blockquote
cite="mid:AANLkTinEDCQWaFnRqjbfqO20xCCmhRQ+RhJz9KptAMs=@mail.gmail.com"
type="cite"><span class="Apple-style-span" style="font-family:
arial,sans-serif; font-size: 13px; border-collapse: collapse;
color: rgb(51, 51, 51);">
<div><span style="font-family: arial,sans-serif; font-size:
13px; border-collapse: collapse; color: rgb(34, 34, 34);">Because
Friendliness is being worked on for very good (safety)
reasons, it seems to me that we should be thinking about the
possibility of "locally Friendly" AGI, just in case
Friendliness is in principle possible, but the full package
SIAI hopes for would just come along too late to be useful.</span></div>
</span></blockquote>
<br>
I do not know of any SIAI push to "universal" Friendliness. Where
do you see this?<br>
<blockquote
cite="mid:AANLkTinEDCQWaFnRqjbfqO20xCCmhRQ+RhJz9KptAMs=@mail.gmail.com"
type="cite"><span class="Apple-style-span" style="font-family:
arial,sans-serif; font-size: 13px; border-collapse: collapse;
color: rgb(51, 51, 51);">
<div><span style="font-family: arial,sans-serif; font-size:
13px; border-collapse: collapse; color: rgb(34, 34, 34);"><br>
</span></div>
<div><span style="font-family: arial,sans-serif; font-size:
13px; border-collapse: collapse; color: rgb(34, 34, 34);">By
"locally Friendly", I mean an AGI that respects certain
boundaries, is Friendly to *certain* people and principles,
but not ALL of them. E.g. a "patriotic American" AGI. That
may sound bad, but if you've got a choice between that and a
completely unconstrained AGI, at least the former would
protect the people it was developed by/for.</span></div>
</span></blockquote>
<br>
And when it encounters the AGI for some other group, what do you
expect to happen?<br>
<br>
- samantha<br>
<br>
</body>
</html>