[ExI] Unfriendly AI is a mistaken idea

[Eugen Leitl]

On Sun, May 27, 2007 at 08:17:35PM +0100, Russell Wallace wrote:

>    In reality however, for better or worse, tools are not persons, nor is
>    there any prospect of it being feasible to create a person from
>    scratch.

So far, you're correct. So far.
 
>      If a piano falls on the top of your head, you're still dead.
>      And no malice intended.
> 
>    In practice, though, I don't actually spend an awful lot of time
>    worrying about the prospect of a piano jumping on my head and killing

Being crushed by a falling piano is typically the cartoon version of
encountering a random event. No malice intended. The same thing happens
when a patch of a rainforest gets turned into a landing strip.

>      Pollution is a killer feature of systems, produced by human
>      engineers.
> 
>    Every system, whether produced by human engineers, evolution or any
>    other source, must produce pollution directly or indirectly, according

Not really, if energy dissipation is negligible (it is but for power plants),
and you have completely closed-loop ecosystems just what the geo/biosphere does,
you're cool.

>    to the laws of thermodynamics. There is therefore no question that an
>    AI will produce pollution (though happily the amount produced per

My pollution was a bad metaphor, think rather habitat restructuring aka
destruction (the biodiversity of a concrete field is rather low).

>    kilowatt, gigaflop or other unit of output is going down - in the
>    Lamarckian evolutionary environment of a man-made technosphere,

Lamarck does hardly exist today. Revisions are pushed out by a
new hardware version, while the old ones wander on the scrap heap
of history.

>    pollution is maladaptive). It doesn't follow that an AI will conquer

An AI won't. Postbiology will.

>    the world. Conquering the world is complex behavior that doesn't
>    happen by accident.

How do you explain we're not a few thousands primate scattered
across Africa? It's a complex behaviour, but there's very little
accident in it.

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE