[ExI] Unfrendly AI is a mistaken idea.

[A B]

Eugen Leitl wrote:

> "I do, I do. Even if such a thing was possible,
you'd
> artificially
> cripple a being, making it unable to reach its full
> potential.
> I'm a religious fundamentalist that way."

But in a sense, aren't all beings in this Universe
"artificially" crippled, in a way? Even a
Universe-Brain will probably hit its limits (but
perhaps not). If I decide to have a child, and I treat
him/her very well, should I still feel guilty about
creating him/her because he/she was unnecessarily
crippled by biological limitations? Some people today
are very happy and content with life, even though they
have the same biological limitations or "crippling".
And, couldn't a Friendly AI still reach its full
potential without destroying humanity? I would like to
reach my full potential too, but my conception of
"potential" doesn't include killing my neighbor and
taking his stuff. If a Friendly AI can still have a
Really^99999999999... good life, but still not be the
*only* mind in the Universe, do you believe that that
is moral grounds for never creating the Friendly AI at
all? - because it will be slightly limited? By the
way, I am sincere with these questions, I'm not just
trying to rile you up. [Or have I just misunderstood
you on this topic?]

Sincerely,

Jeffrey Herrlich 


--- Eugen Leitl <eugen at leitl.org> wrote:

> On Wed, Jun 13, 2007 at 05:21:01PM +1000, Stathis
> Papaioannou wrote:
> 
> >    I'd rather that the AI's in general *didn't*
> have an opinion on
> >    whether it was good or bad to harm human
> beings, or any other opinion
> >    in terms of "good" and "bad". Ethics is
> dangerous: some of the worst
> 
> Then it would be very, very close to being
> psychpathic
>
http://www.cerebromente.org.br/n07/doencas/disease_i.htm
> 
> Absense of certain equipment can be harmful.
> 
> >    monsters in history were convinced that they
> were doing the "right"
> >    thing. It's bad enough having humans to deal
> with without the fear
> >    that a machine might also have an agenda of its
> own. If the AI just
> 
> If you have an agent which is useful, it has to
> develop its own
> agendas, which you can't control. You can't
> micromanage agents; orelse
> making such agents would be detrimental, and not
> helpful.
> 
> 
> >    does what it's told, even if that means killing
> people, then as long
> >    as there isn't just one guy with a super AI (or
> one super AI that
> 
> There's a veritable arms race on in making smarter
> weapons, and
> of course the smarter the better. There are few
> winners in a race,
> typically just one.
> 
> >    spontaneously develops an agenda of its own,
> which will always be a
> >    possibility), then we are no worse off than we
> have ever been, with
> >    each individual human trying to get to step
> over everyone else to get
> >    to the top of the heap.
> 
> With the difference that we are mere mortals,
> competing among themselves.
> A postbiological ecology is a great place to be, if
> you're a machine-phase
> critter. If you're not, then you're food.
> 
> >    I don't accept the "slave AI is bad" objection.
> The ability to be
> 
> I do, I do. Even if such a thing was possible, you'd
> artificially
> cripple a being, making it unable to reach its full
> potential.
> I'm a religious fundamentalist that way.
> 
> >    aware of one's existence and/or the ability to
> solve intellectual
> >    problems does not necessarily create a
> preference for or against a
> >    particular lifestyle. Even if it could be shown
> that all naturally
> >    evolved conscious beings have certain
> preferences and values in
> >    common, naturally evolved conscious beings are
> only a subset of all
> >    possible conscious beings.
> 
> Do you think Vinge's Focus is benign? Assuming we
> would engineer
> babies to be born focused on a particular task,
> would you think it's
> a good thing? Perhaps not so brave, this new
> world...
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
>
http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
> 



       
____________________________________________________________________________________
Choose the right car based on your needs.  Check out Yahoo! Autos new Car Finder tool.
http://autos.yahoo.com/carfinder/