[extropy-chat] Singularity econimic tradeoffs

[Samantha Atkins]

On Apr 18, 2004, at 7:30 PM, Dan Clemmensen wrote:

>>> In my opinion, the Singularity will result from any fast-feedback 
>>> process that uses computers to enhance "technological creativity." 
>>> "Technological creativity"is that quality or set of qualities that 
>>> results in new advances in technology. For purposes of this 
>>> discussion, we can restrict ourselves to computer and related 
>>> technologies.
>>>
>>> In my favorite scenario, the initial SI is a collaboration between 
>>> one or more humans and a large computing resource. The humans supply 
>>> the creativity and high-level pattern recognition, while the 
>>> computers supply brute-force stuff like web searching, peep-hole 
>>> optimizations, etc. If the collaboration can be made tight enough, 
>>> the system as a whole will operate as fast as the human(s) can make 
>>> high-level decisions.
>>
>>
>> Bingo!   Since humans are making the high-level decision the 
>> speed/productivity of the system is directly limited by human 
>> limitations.  The include human irrationality and various forms of 
>> monkey politics inherent to human group dynamics.
>
>
> When decision criteria are presented to a human in an understandable 
> form, the human can make decisions very rapidly. The decisions a 
> racing card driver, a fighter pilot, or a video gamer makes are a case 
> in point. This is decision-making on an entirely different level that 
> the one we generally think of in the business world. A programmer 
> currently makes a few creative decisions per day at the most. The rest 
> of the programming job consists of implementing those decisions by 
> writing, compiling, debugging, and releasing code.
>

Notice that all your examples are in realms of high hand-eye 
coordination.  Humans were optimized in those areas by evolution.  I 
see no evidence that similar high speed decision making is achieved by 
humans, individually or collectively, in dense informational contexts 
that we were not optimized for.

>>>  Such a system would permit computer implementation under human 
>>> guidance. Presumably, the first thing the inventors of such a system 
>>> will use the system for is improvements to the system.
>>
>>
>> I have been around in corporate/business computing environments for 
>> quite some time and I have been involved in business selling to such 
>> environments and improving productivity of various groups, including 
>> software groups.  The first and foremost thing any IT resource is 
>> used for is to enhance profitability.   Self-improvement of IT 
>> systems,  done in-house, by external software purchase and 
>> integration or some mixture are usually not a very high priority.  It 
>> is notoriously hard to sell software based on such improvements to 
>> infrastructure.  Usually the improvements have to be cast in terms of 
>> "solutions" to particular onerous problems seen as part of the 
>> business process directly influencing the bottom line.  This 
>> recasting as solutions severely limits how much improvement is 
>> achieved or even contemplated to fundamental infrastructure and 
>> methodology.
>
> We are at different levels here. I'm not talking about IT departments. 
> I'm talking about an individual hacker, or perhaps a researcher. I'm 
> also not talking about top-down "productivity" tools like Rational 
> Rose. I'm talking more of bottom-up tools like the various IDEs.

I have also been hacking for a lot of years.  When I am fully on I am 
considered " scary" by most hacker's standards.  But I am also quite 
aware of how slow my best speed and how limited my best "flow" is in 
the context of what is needed.   IDEs?    Bottom-up?   <stunned>

>
>>
>> It is even more difficult to make a profitable business selling tools 
>> to software producers, i.e., programmers.  The business model just 
>> doesn't work out that well.  It is possible to pay the bills of a 
>> small company and group in this manner but there isn't much way to 
>> get rich doing this.  Yet the problems of improving software 
>> productivity and quality are very germane to such a path to SI and 
>> are largely non-trivial problems.  Open Source efforts hold some 
>> promise but I have my doubts the most central problems will be solved 
>> in the OS world.
>>
>>> As soon as the system is implementing things as fast as the human(s) 
>>> can make decisions, the next problem that the inventors will turn to 
>>> is increasing the scope of sub-problems that can be solved by the 
>>> computers rather than the human, using whatever software tools come 
>>> to hand: there is no particular need for an overall theory of AI 
>>> here. since the humans are still handling that part. The humans 
>>> become more and more productive. As they add more and more tools to 
>>> the computer toolbox, the humans operate at progressively higher 
>>> levels of abstraction. They use the system to optimize the system. 
>>> If necessary, they use the system to design new hardware to add to 
>>> the system. Eventually, the humans are operating at such a high 
>>> level of abstraction that the non-human part of the system reaches 
>>> and then exceeds the current human level of technical creativity.
>>
>>
>> There is a very real Ai component needed for such planning, 
>> scheduling, understanding intent of decisions, weighing repercussions 
>> of implementation choices, deciding when to bring humans back into 
>> the loop and so on.  This is very non-trivial and not at all in the 
>> scope of most business computing today.
>
> People insist on trying to computerize the wrong parts of the problem. 
> You speak of planning, scheduling, and understanding. I don't want the 
> computer to do these tasks, at least not initially. I want the 
> computer to do the mundane stuff, it can ask ME to do the planning. If 
> the computer element cannot "understand the intent" it should ask me. 
> The computer can indeed "weight the implications" of certain 
> decisions, by simply executing all the branches and displaying the 
> results. However if such an activity is too expensive, the computer 
> should tell me and ask for another decision.
>

Humans are too limited to do real time planning at a fine grained level 
quickly enough to remotely keep even rather modest hardware busy with 
the rest of your wish list above.   The computer also needs to be able 
to analyze the various runs to at least a well-chewed summary level.  
There is no way a human can analyze more than a relative handful of 
runs in sufficient detail to guide further work quickly enough to 
better even outstanding human level results, much less achieve 
Singularity levels.

>>
>> Having humans operate at such a high level of abstraction is not at 
>> all easy to do.  Only persons well trained in formal abstract 
>> reasoning are likely to comfortably operate at such levels and then 
>> only within the severe limits set by our internal computational 
>> hardware and biologically heavily conditioned minds.  Their is some 
>> truth in the old chestnut that groups of humans often have an 
>> effective intelligence no greater than 70% of the average 
>> intelligence of the group.  I am being charitable when I say this is 
>> what I have observed.  Much of the group decision making is not on 
>> the basis of the manipulation of the principles and abstractions 
>> involved using logic at all.
>
> I have seen a five-year-old child making decisions at a high level in 
> real time, on the soccer field, with two days of training.

Again, proficiency in monkey skills does not translate to general high 
speed abstract decision making.

>  The kid will obviously not make soccer decisions at the same level as 
> a professional, but the level is still higher than that of a computer. 
> This particular kid has never received formal training in abstract 
> reasoning.

Irrelevant as that is not what is used to play soccer.

>
>
> It appears that we are in complete agreement. The seed SI has a human 
> component, this seed SI bootstraps, and eventually (within a week in 
> my scenario) reaches a level at which the human component is no longer 
> useful.  My point is that un-augmented humans do not need to create an 
> AI to reach this point.
>

I got your point.  I still disagree.

-s