Structure of AI (was: Re: [extropy-chat] COMP: Distributed Computing)

[J. Andrew Rogers]

On Nov 21, 2004, at 11:21 AM, Samantha Atkins wrote:
> On Nov 20, 2004, at 2:09 PM, J. Andrew Rogers wrote:
>> Or to put it another way, there is no theoretical reason that one 
>> could not create a human-level intelligence on an old 486 computer.  
>> The speed of the computer is orthogonal to the problem space, and 
>> only weakly relevant as a practical matter.
>
> Go ahead then.  If by human-level intelligence you mean remotely 
> equivalent in terms of cognitive operations per unit time then I would 
> be very surprised if any such thing could be acheived on such an 
> architecture.   I don't see that the grater switching speed vs lack of 
> significant parallelism is in favor of such.


As a practical matter I wouldn't want to do anything on a 486, if only 
because the memory systems it supports are terribly inadequate.  You 
could, but you wouldn't want to, though a simple and slow core by 
today's standards should provide plenty of IPC if the rest of the 
architecture was in order.  There were two points here:

1.)  There is nothing in "intelligence" that has a time dimension in 
the theoretical.  In any finite context, there is no "intelligence per 
unit time" that reflects on the intrinsic intelligence of the system 
being measured.  For any time-bounded intelligence metric you can think 
of, there is a "fast and stupid" machine that will appear more 
intelligent than a "slow and smart" machine, for the purposes of black 
box comparison.  Of course, the point of AI is to come up with an 
algorithm that will be smart in any context, not to game intelligence 
metrics.

2.)  If you have a system that has O(log n) complexity in one dimension 
and O(n^k) or O(k^n) complexity in another dimension, the latter 
function will dominate scaling for some non-trivial "n".  For systems 
that must express some approximation of universal induction, a general 
mathematical requirement for AI, space complexity is a problem that 
dwarfs time complexity.  An extreme and relevant example of this in 
literature is the sequential universal predictor algorithm -- 
logarithmic traversal time, geometric space (hence its intractability). 
  On real silicon, this severely bottlenecks on memory latency, since 
you only need to dispatch a handful of machine code instructions per 
memory address.  It becomes literally the case that a few GB worth of 
patterns can be exhaustively searched using fewer clock cycles than are 
wasted in a single memory stall, and with data structures of this type, 
cache misses are the rule.  Practical AI will have to have a similar 
complexity profile as a consequence of approximating some mathematical 
requirements, though obviously not as severe.

cheers,

j. andrew rogers