[ExI] the ethics of the Vile Offspring

[Keith Henson]

On Sun, May 22, 2011 at 7:28 AM, Anders Sandberg <anders at aleph.se> wrote:
> Stefano Vaj wrote:
>>
>> Conversely, take Stross's Accelerando. How should we condone the
>> incredibly parochial hostility to what is cavalierly defined the Vile
>> Offspring which is expressed in the last part of the book?
>
> Moral relativists or people viewing value as being a purely social or
> psychological construct  have no problem in accepting the character's
> hostility to the VO. But it might be a correct ethical judgement even if
> there exists an absolute standard for value.
>
> I suspect Stross got the idea partially from Nick Bostrom's paper "The
> future of human evolution" where he discusses scenarios where posthumanity
> evolves into something that completely lacks whatever it is that actually
> gives existence value, for example a very capable and expansive civilization
> where there is no consciousness.

I get the impression that Stross is modeling the VO after an ecosystem
of corporations which fit "a very capable and expansive civilization
where there is no consciousness."  Or maybe no conscious or both.

> (Stross may however independently have had
> a similar idea, there is something similar in his early work "Scratch
> Monkey"). The Vile Offspring might simply lack any form of value (moral,
> aesthetic, etc) and be an unstoppable force squeezing out systems that do
> have value.

They live off sunlight and are in the final stages of disassembling
the planets to capture it.  But that activity is a smaller fraction of
their economy than food is to current day humans.  They are able to
produce human like entities from anyone who left a corpus of written
works.  I think Stross got that idea right here on this list where I
was promoting cryonics and Hans Moravec said he didn't need it because
he could be reconstructed from his works.  I argued back that it would
waste truly vast computational resources and involve discarding
millions of versions that didn't come close enough.

To see that early Extropian list exchange skillfully turned into a
plot element in Accelerando is so cool it makes my head spin.  Charles
may still be on this list, but it's been years since he said anything.
 (Not that I want him to, rather he spent his time writing more really
great stuff, _Rule 34_ is due out this summer.)

> Of course, humans or human-derived beings might just parochially think
> anything incompatible with their mode of existence and their values to be
> valueless - it could just as well be that the VO is actually fantastically
> good by some unknown absolute standard (and that the best thing any
> posthuman can do is to help it grow). But it is not obvious that a randomly
> evolved or designed thing must strive towards maximal value by this standard
> (the paperclip maximizer AI is a fine counterexample), or even that it is
> very likely.

The VO are obviously trying to turn everything in reach of light into
computronium.  Humans can live human style lives in that state, that's
how they traveled by laser cannon.  It is, however, a cockroach sort
of existence, always in danger of being stomped out.

Keith