[ExI] the ethics of the Vile Offspring
Anders Sandberg
anders at aleph.se
Sun May 22 16:19:24 UTC 2011
Stefano Vaj wrote:
> > I suspect Stross got the idea partially from Nick Bostrom's paper
> "The future of human evolution" where he discusses scenarios where
> posthumanity evolves into something that completely lacks whatever it
> is that actually gives existence value, for example a very capable and
> expansive civilization where there is no consciousness.
>
> Yes, so do I. In fact, I was simply flabbergasted when I was first
> exposed to Bostrom's inclination to justify what is exactly the
> argument against trans-simianism by resorting to qualia.
He is not basing it on qualia, he is using qualia as an example. Maybe
the real value resides in something else, but his point still stands:
the kind of evolution we might engage in in the future might push us
away from whatever the real value is.
If we think evolution has so far pushed us in the direction of value it
does not follow that future new kinds of evolution will continue to push
in the right way.
It might of course be that we now, being better aware of value, can push
our evolution in an even more desireable direction. But given past
experience with human planning and coordination ability for complex
social systems as well as our wide spread of opinion on what is
valuable, this does not look guaranteed in any way. In fact, such
projects might develop their own accidental dynamics pushing in the
wrong way (like many of our social institutions do).
The problem is not the overman, the problem is that it might be rational
for everybody to become something that in the aggregate or individually
has lower value.
Monkeys might disagree with us about what is really valuable in life and
might think that we have evolved in a direction that produces less
value. They could just be wrong about that, since we have sources of
value they lack access to, like science, philosophy and culture. But
that doesn't mean some forms of future evolution can't be value-eroding.
Robin's paper on the cosmic commons points out one nasty possibility
(turning into an interstellar locust swarm) that might be hard to avoid
without rather serious coordination (perhaps necessitating other,
dangerous social institutions).
--
Anders Sandberg,
Future of Humanity Institute
James Martin 21st Century School
Philosophy Faculty
Oxford University
More information about the extropy-chat
mailing list