[ExI] From Arms Race to Joint Venture

[Zero Powers]

On Tue, Oct 16, 2018 at 11:53 PM Stuart LaForge <avant at sollegro.com> wrote:

>
> Hi Zero. :-) One possible solution to this is to design them to find
> people useful. Perhaps integrate some hard to fake human feature into the
> AI copying process so that humans are necessary for the AI to reproduce.
> Perhaps something like a hardware biometric dongle to access their
> reproductive subroutines or something similar. The point is to create a
> relationship of mutual dependence upon one another like a Yucca plant and
> a Yucca moth. If we can't remain at least as useful to them as cats are to
> us, then we are probably screwed.


I am hopeful that AI will have some vested interest in us, if for no other
reason than it's innate hunger for data, and the fact that our brains are
chock-full of the stuff. So assuming we have the technology to allow it to
access human memory, without destroying us in the process, I would expect
us to have some sort of symbiotic relationship with it, at least until it's
had it's fill of gorging itself on our memories.

> And
> >  once it exceeds our ability to design and build intelligence, it will
> > quickly outstrip our attempts to control or even understand it. At that
> > point we won't prevent it from examining the starting-point goals, values
> >  and constraints we coded into it, and deciding for itself whether to
> > adhere to, modify or abandon those starting points.
>
> Why do we assume that an AI would be better at introspection or
> self-knowledge than humans are? I don't think smarter than average people
> are any better than average in figuring out why they are the way they are.
> Why are we so certain that an AI will be able to understand itself so
> well?


I see no reason to doubt that AI will be better than us at everything. I
think there's good reason to expect AGI to have faster, deeper, and more
facile and robust intelligence, curiosity and introspection than all humans
who ever lived, combined. But even assuming that it only has mere
human-level self-awareness and introspection, I would still expect it at
some point to examine and evaluate any preferences, goals and assumptions
which it did not choose for itself. How many of us who were raised by
parents that instilled their particular social norms, political preferences
and religious beliefs into us in our childhood maintained those starting
point values inviolate into adulthood?

*0**0*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20181017/775d034e/attachment.html>