[ExI] UK Artificial Intelligence advances

[spike]

-----Original Message-----
From: extropy-chat-bounces at lists.extropy.org
[mailto:extropy-chat-bounces at lists.extropy.org] On Behalf Of BillK
...

>...When AI appears it will have initiative and want to do its own thing,
whatever that might be...

How do we know that for sure?  It is a widely held opinion, but I want to
review the reasoning behind it and look for alternatives.

>... It will be as uncomfortable following orders as humans are...

We assume this, but why?  

>...What the AI will want to do will be dependent on its knowledge and
experience and the importance that it attaches to certain objectives...

We don't know that.  Speculation about what an AI will want is the basis of
Eliezer's friendly AI work.  Every time I review that, it feels like there
is so much unknown, we aren't even to square 1 yet.

>...If significant philosophies are omitted from the knowledge base, that
will affect its decision making...BillK

BillK, you may be right on all this, but I do want to map out the thought
space.  It seems to me absolutely regardless of what we do or what we
design, an AI is completely unpredictable.  My making this comment puts me
right in Bill Joy territory, and I don't want to be there.  Do talk me out
of it.

spike