[ExI] Lethal future was Watson on NOVA

[Kelly Anderson]

On Fri, Feb 18, 2011 at 5:51 AM, Eugen Leitl <eugen at leitl.org> wrote:
> On Fri, Feb 18, 2011 at 12:10:41AM -0700, Kelly Anderson wrote:
>
>> > Look, what is your energetical footprint? 1 kW, more or less?
>> > Negligible.
>>
>> In a super efficient system, my footprint might be nanowatts. I
>
> Not even for human equivalent, nevermind at 10^6 to 10^9 speedup.
> I don't think you can go below 1-10 W for a human realtime equivalent.

Are you assuming the use of today's technology? Or the best that may
be created in the future?

According to several sites on the Internet the human brain uses 20-40
Watts. Some of that undoubtedly goes for biological purposes that are
not directly supportive of computation.

It seems very pessimistic to say that we could only improve by 2-40
times over nature. Granted nanowatts may be overly optimistic, and is
based on no currently known technology. Nevertheless, I see no reason
to believe that that the bottom is 1 Watt.

>> believe there are theoretical computing models that use zero net
>> electricity.
>
> Reversible logic is slow, and it's not perfectly reversible.

Not yet, of course. And human brains are very slow too.

An interesting question to be answered is what is the most limiting
factor? Is it matter out of which to build intelligence? Is it energy
to power it? Time to run it? Or space to house it? Or is there some
other limiting factor? I think it will take a while for the
exponential growth to stop, but it must eventually stop. I'm just not
sure which of the above is the most limiting factor. Only time and
technology will tell. I'm not sure we can even guess at this point
what the most limiting factor will be.

When the actual limiting factor is determined, then Darwinism will
kick in and we'll see what the results are.

> And it's still immaterial, because if you use 100 times less energy
> there will be 100 times the individuals competing for it. Adaptively.

Software is a gas. I have understood that for a very long time.

>> > Now multiply that by 7 gigamonkeys. Problem?</trollface>
>> >
>> > Infinitesimally small energy budgets multiplied by very large
>> > numbers are turning stars into FIR blackbodies. And whole galaxies,
>> > and clusters, and superclusters.
>> >
>> > You think that would be easy to miss?
>>
>> Yes. Seeing the LACK of something is very difficult astronomy. Heck
>
> Giant (up to GLYr) spherical voids only emitting in FIR?

You are probably right on this one. I hadn't thought that through and
I don't have a lot of experience with the technical aspects of
astronomy.

>> how long did it take astronomers to figure out that the majority of
>> the universe is dark matter? I agree with you that an advanced
>
> There was a dedicated search for Dyson FIR emitters. Result:
> density too low to care.

OK. I am sure we all would have heard on the evening news had
something like that been located.

>> civilization would eventually create a ring world, and finally a
>
> Not ring, optically dense node cloud.

Most likely.

>> sphere that collected all available solar energy. But that could
>> support an enourmous computational structuree, capable of simulating
>
> Enormous to some, trivial to others.
>
>> every mind in a 10,000 year civilization might take only a few watts
>> and a few seconds.
>
> The numbers don't check out. Occam's razor sez: we're not in anyone's
> smart lightcone.

The proof of extraterrestrial life wasn't the fundamental portion of
my argument. Closer to the center is that with human psychology
running the show, many, if not most, people would likely retreat into
VR given the choice.

>> > When something is postulated to you it's usually bunk. Novelty
>> > and too small group for peer review pretty much see to that.
>>
>> When I look at teenagers lost in iPods, it doesn't seem like bunk to
>> think that they could positively be swallowed alive by an interesting
>> virtual reality. I have relatives who have addiction to WoW that makes
>> a heroin addict look like a weekend social drinker.
>
> Have you seen the birth rate and retention rate of Amish?

No. What is your point? Really, I don't understand what you're implying.

>> growth cannot continue forever indicates that there will be Darwinian
>> processes for choosing which AGIs get the eventually limited power,
>
> You're getting it.
>
>> and which do not. This leads one inevitably to the conclusion that the
>> surviving AGIs will be the "fittest" in a survival and reproduction
>> sense. It will be a very competitive world for unenhanced human beings
>> to compete in, to say the least.
>
> Exactly.

I don't hold out a lot of optimism for humans as they now exist. I
hold out a LOT of optimism for our progeny (whatever form that progeny
takes).

Responding to the thread as a whole, Darwinian forces in the future
will probably be more focused on the replication of temes, (techno
memes) rather than physical reproduction. Memes and temes reproduce
much faster than genes, and overpower the slower replicators without
much effort (Dawkins, Blackmore). No matter how smart the machines are
in the future, no one of them will know everything that all the others
know, and so there will always remain a necessity for some kind of
networking or shared memory. Even viewed as one super organism, there
are organelles or cells that contain unique information that will want
to be copied and processed.

If some kind of economy survives the singularity, then some
information will still have some value that will have to be paid for
by some form of money. I haven't seen much written to this point on
money and economics in the coming Singularity. Have I missed
something?

-Kelly