[extropy-chat] Singularity econimic tradeoffs

[Dan Clemmensen]

Samantha Atkins wrote:

>
> On Apr 17, 2004, at 1:32 PM, Dan Clemmensen wrote:
>
>> Samantha Atkins wrote:
>>
>>>>
>>> Please clarify how increased computer power and (supposedly non-AI) 
>>> software will lead to a singularity.
>>>
>> OK!
>>
>> In my opinion, the Singularity will result from any fast-feedback 
>> process that uses computers to enhance "technological creativity." 
>> "Technological creativity"is that quality or set of qualities that 
>> results in new advances in technology. For purposes of this 
>> discussion, we can restrict ourselves to computer and related 
>> technologies.
>>
>> In my favorite scenario, the initial SI is a collaboration between 
>> one or more humans and a large computing resource. The humans supply 
>> the creativity and high-level pattern recognition, while the 
>> computers supply brute-force stuff like web searching, peep-hole 
>> optimizations, etc. If the collaboration can be made tight enough, 
>> the system as a whole will operate as fast as the human(s) can make 
>> high-level decisions.
>
>
> Bingo!   Since humans are making the high-level decision the 
> speed/productivity of the system is directly limited by human 
> limitations.  The include human irrationality and various forms of 
> monkey politics inherent to human group dynamics.


When decision criteria are presented to a human in an understandable 
form, the human can make decisions very rapidly. The decisions a racing 
card driver, a fighter pilot, or a video gamer makes are a case in 
point. This is decision-making on an entirely different level that the 
one we generally think of in the business world. A programmer currently 
makes a few creative decisions per day at the most. The rest of the 
programming job consists of implementing those decisions by writing, 
compiling, debugging, and releasing code.

>>  Such a system would permit computer implementation under human 
>> guidance. Presumably, the first thing the inventors of such a system 
>> will use the system for is improvements to the system.
>
>
> I have been around in corporate/business computing environments for 
> quite some time and I have been involved in business selling to such 
> environments and improving productivity of various groups, including 
> software groups.  The first and foremost thing any IT resource is used 
> for is to enhance profitability.   Self-improvement of IT systems,  
> done in-house, by external software purchase and integration or some 
> mixture are usually not a very high priority.  It is notoriously hard 
> to sell software based on such improvements to infrastructure.  
> Usually the improvements have to be cast in terms of "solutions" to 
> particular onerous problems seen as part of the business process 
> directly influencing the bottom line.  This recasting as solutions 
> severely limits how much improvement is achieved or even contemplated 
> to fundamental infrastructure and methodology.

We are at different levels here. I'm not talking about IT departments. 
I'm talking about an individual hacker, or perhaps a researcher. I'm 
also not talking about top-down "productivity" tools like Rational Rose. 
I'm talking more of bottom-up tools like the various IDEs.

>
> It is even more difficult to make a profitable business selling tools 
> to software producers, i.e., programmers.  The business model just 
> doesn't work out that well.  It is possible to pay the bills of a 
> small company and group in this manner but there isn't much way to get 
> rich doing this.  Yet the problems of improving software productivity 
> and quality are very germane to such a path to SI and are largely 
> non-trivial problems.  Open Source efforts hold some promise but I 
> have my doubts the most central problems will be solved in the OS world.
>
>> As soon as the system is implementing things as fast as the human(s) 
>> can make decisions, the next problem that the inventors will turn to 
>> is increasing the scope of sub-problems that can be solved by the 
>> computers rather than the human, using whatever software tools come 
>> to hand: there is no particular need for an overall theory of AI 
>> here. since the humans are still handling that part. The humans 
>> become more and more productive. As they add more and more tools to 
>> the computer toolbox, the humans operate at progressively higher 
>> levels of abstraction. They use the system to optimize the system. If 
>> necessary, they use the system to design new hardware to add to the 
>> system. Eventually, the humans are operating at such a high level of 
>> abstraction that the non-human part of the system reaches and then 
>> exceeds the current human level of technical creativity.
>
>
> There is a very real Ai component needed for such planning, 
> scheduling, understanding intent of decisions, weighing repercussions 
> of implementation choices, deciding when to bring humans back into the 
> loop and so on.  This is very non-trivial and not at all in the scope 
> of most business computing today.

People insist on trying to computerize the wrong parts of the problem. 
You speak of planning, scheduling, and understanding. I don't want the 
computer to do these tasks, at least not initially. I want the computer 
to do the mundane stuff, it can ask ME to do the planning. If the 
computer element cannot "understand the intent" it should ask me. The 
computer can indeed "weight the implications" of certain decisions, by 
simply executing all the branches and displaying the results. However if 
such an activity is too expensive, the computer should tell me and ask 
for another decision.

>
> Having humans operate at such a high level of abstraction is not at 
> all easy to do.  Only persons well trained in formal abstract 
> reasoning are likely to comfortably operate at such levels and then 
> only within the severe limits set by our internal computational 
> hardware and biologically heavily conditioned minds.  Their is some 
> truth in the old chestnut that groups of humans often have an 
> effective intelligence no greater than 70% of the average intelligence 
> of the group.  I am being charitable when I say this is what I have 
> observed.  Much of the group decision making is not on the basis of 
> the manipulation of the principles and abstractions involved using 
> logic at all.

I have seen a five-year-old child making decisions at a high level in 
real time, on the soccer field, with two days of training. The kid will 
obviously not make soccer decisions at the same level as a professional, 
but the level is still higher than that of a computer. This particular 
kid has never received formal training in abstract reasoning. However, 
for the sake of argument, let's assume it does in fact require a trained 
decision-maker. Given that we only need one of these people to bootstrap 
the SI, this is not much of a barrier. Pleas consider the amount of time 
it takes a new player to become proficient at a complex video game. A 
game player is essentially doing nothing but making decisions.

>>
>> The richer the available computer resource the faster this will go. 
>> My gut feeling is that the currently-available computer resources are 
>> already rich enough that the process no linger needs new hardware to 
>> go to completion, and can therefore go to completion in less than one 
>> week, at which point all connected computes form a single 
>> fully=optimized system, which has also designed and sent fabrication 
>> and purchase orders for the next-generation hardware and for the 
>> equipment needed to produce nanotech.
>>
>
> In this scenario the determining factor is the rationality and ability 
> to abstract effectively of the human component.   I very much agree 
> there is a lot of promise in the power of IA to augment the range, 
> creativity and abilities of human individuals and groups.  But such 
> power is not likely IMHO to play more than a bootstrapping role in the 
> creation of true SI.

It appears that we are in complete agreement. The seed SI has a human 
component, this seed SI bootstraps, and eventually (within a week in my 
scenario) reaches a level at which the human component is no longer 
useful.  My point is that un-augmented humans do not need to create an 
AI to reach this point.