[ExI] UK Artificial Intelligence advances

[BillK]

On Wed, Aug 14, 2013 at 10:53 PM, spike wrote:
> How do we know that for sure?  It is a widely held opinion, but I want to
> review the reasoning behind it and look for alternatives.
>

If it doesn't have initiative then it isn't AI. It is a clever
question and answer machine, like Watson.


> We don't know that.  Speculation about what an AI will want is the basis of
> Eliezer's friendly AI work.  Every time I review that, it feels like there
> is so much unknown, we aren't even to square 1 yet.
>
> BillK, you may be right on all this, but I do want to map out the thought
> space.  It seems to me absolutely regardless of what we do or what we
> design, an AI is completely unpredictable.  My making this comment puts me
> right in Bill Joy territory, and I don't want to be there.  Do talk me out
> of it.
>


The trouble is that it isn't completely unpredictable. If the military
want a homicidal, but brilliant, military commander. then they can
probably get one by building an organizing intelligence on top of a
combination of military philosophies. They will start with a question
/ answer system, but some situations will require initiative without
waiting for human instruction. Because faster response wins the
conflict. And the enemy might be building their own military AI.

A bleak future indeed.

BillK