[ExI] The symbol grounding problem in strong AI

[Emlyn]

2009/12/15 Eugen Leitl <eugen at leitl.org>:
> On Tue, Dec 15, 2009 at 02:34:39PM +1030, Emlyn wrote:
>
>> This is the real answer to the "consciousness" problem, imo. You will
>> know if AI is conscious because you'll just ask it if it is, and
>
> I don't know what "conscious" even means, but you know AI has achieved
> full human equivalence across the board once everyone is out of job.

Only equivalence in the most banal sense. My feeling is that machine
intelligence will replace us well before it's "general intelligence",
whatever that means. We're not even general intelligences.

>
>> you'll be able to observe its behaviour and see if is influenced by
>> its own sense of consciousness. The problem of whether it is telling
>> the truth is identical to the problem of whether people lie about this
>> now; you can't know and it doesn't matter.
>>
>> Most likely, an AI which is not an emulation of evolved biology will
>
> I would not be holding my breath for that one.

True. (exhales)

>
>> experience something entirely unlike what we experience. It should be
>> pretty damned interesting, and illuminating for humanity, to interact
>> with such alien critters!
>
> The first generations are human demand-driven, and hence perfectly
> boring.

Well, in the it-can't-kill-me-yawn kind of boring. I'm sure they'll be
interesting, just like a quicksort is interesting.

> The other kind is incomprehensible and/or lethal, so I'm
> not sure I would want to talk to them unless I'm them. Then you
> don't want to talk to me.

I always want to talk to you Eugen. But then you can never be them,
can you? None of us can, by definition.

But incomprehensible, not so! It's very likely that if we can build an
AGI without basing it on biology, we'll be able to understand it in
principle far better than we can understand our own workings (which do
seem to be a little bit convoluted). That's not to say that you'd want
your daughter marrying one...

-- 
Emlyn

http://emlyntech.wordpress.com - coding related
http://point7.wordpress.com - ranting
http://emlynoregan.com - main site