[extropy-chat] Engineered Religion-- Your Mom and the Machine

[Robin Hanson]

At 12:45 PM 3/25/2005, john-c-wright at sff.net wrote:
>Ben, whose sense of humor is no better and no worse than my own, poses a
>significant epistimological question. He is asking on what grounds the 
>faithful
>(as we call ourselves) or foam-at-the-mouth zealots in teapot-shaped hats (as
>others call us) take certain things on faith, or on authority, or as a 
>result of
>innate knowledge or revelation? How does one distinguish a valid authority 
>from
>an invalid, or tell a true intuition from a false?
>I cannot imagine that the majority of the posters here would be interested 
>in a
>discussion of this kind, which is a straight-up philosophical question, black
>without cream or sugar. I will be happy to write him privately with my answer,
>such as it is.

I'll bet there are in fact enough of us interested in that sort of 
conversation to
make it worth having here.  I know I am, at least if it is at a high level.

I don't see why your machine needs to have a "higher rule" than empiricism 
so that
it will not destroy the world.  I just has to not *want* to destroy the 
world.
It doesn't need to believe in morality, or consciousness, or a "Mother rule".
You seem to imagine that certain wants are natural and dangerous, and so 
creatures
must be built to follow Mother rules which overrule those natural wants or all
may be lost.  I can see a concern about the wants that evolutionary selection
would produce, but if we are just talking about creating a single creature 
I don't
see why it can't be just made not to want to destroy the world.





Robin Hanson  rhanson at gmu.edu  http://hanson.gmu.edu
Assistant Professor of Economics, George Mason University
MSN 1D3, Carow Hall, Fairfax VA 22030-4444
703-993-2326  FAX: 703-993-2323