[ExI] The symbol grounding problem in strong AI

[Eugen Leitl]

On Tue, Dec 15, 2009 at 09:35:46PM +1030, Emlyn wrote:

> > I don't know what "conscious" even means, but you know AI has achieved
> > full human equivalence across the board once everyone is out of job.
> 
> Only equivalence in the most banal sense. My feeling is that machine

Do not underestimate activities people will pay money for. Many
of them tackle people to their very limit. No longer competitive across
the board is a pretty taxing benchmark.

> intelligence will replace us well before it's "general intelligence",
> whatever that means. We're not even general intelligences.

We're as good a yardstick as anything. You need to define an origin somewhere.
 
> Well, in the it-can't-kill-me-yawn kind of boring. I'm sure they'll be
> interesting, just like a quicksort is interesting.

Watching this beats most TV programming, I guess.
 
> I always want to talk to you Eugen. But then you can never be them,
> can you? None of us can, by definition.

Biology-derived systems are also capable of self-enhancement runaways.
Technically you is a moving target, and there's a continuous trajectory
all along the way, but enough quantity will turn into quality. You don't
need a lot of delta to be completely incomprehensible. You can easily
see it in some people already.
 
> But incomprehensible, not so! It's very likely that if we can build an
> AGI without basing it on biology, we'll be able to understand it in

If you do it by a darwinian design (nobody so far seems to get a handle on
how to do it any other way) it's just as opaque. There's modularity for
functional compartment in the body, but just not a lot of it in the brain.
Best make modularity part of the fitness function.

> principle far better than we can understand our own workings (which do
> seem to be a little bit convoluted). That's not to say that you'd want
> your daughter marrying one...

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE