[extropy-chat] Engineering Religion

[Robin Hanson]

At 04:30 PM 3/21/2005, john-c-wright at sff.net wrote:
>If you write into their basic program, let us say, a philosophy of 
>behaviorism,
>then they should end up believing that everything they believe is due to their
>programming. If, on the other hand, you write into their basic program a
>philosophy that believes in free will, I suppose they will either end up
>believing in free will, or not, as they chose.
>
>Likewise, if the Jupiter Brain Engineers download the entire history of human
>religious thought, and then write into the Brains' basic program that they
>should adopt a rule of evidence to dismiss this all as delusion and fraud, you
>will probably get a different result than if you write into their basic 
>program
>a belief in intuition or inspiration, or a rule of evidence that places weight
>on authority and tradition. God knows what would happen then.

I'm with Eliezer here - Jupiter Brains should just not be that stupid.  If you
knew that the people who built your brain had a certain agenda when they
started you, you will want to correct for that bias as best you could.  The
fact that your creators wanted you to believe certain things might itself be
taken as evidence for the truth of those things, but otherwise you should
not want your beliefs to depend their manipulations.

In fact, I have a forthcoming paper on exactly this topic:
----------------------------------------------------------------------------
forthcoming in Theory and Decision, 2005.
http://hanson.gmu.edu/prior.pdf

                  Uncommon Priors Require Origin Disputes
                             by Robin Hanson

In standard belief models, priors are always common knowledge.  This prevents
such models from representing agents' probabilistic beliefs about the origins
of their priors.  By embedding standard models in a larger standard model,
however, *pre-priors* can describe such beliefs.  When an agent's prior and
pre-prior are mutually consistent, he must believe that his prior would only
have been different in situations where relevant event chances were different,
but that variation in other agents' priors are otherwise completely unrelated
to which events are how likely. Thus Bayesians who agree enough about the
origins of their priors must have the same priors.
----------------------------------------------------------------------------



Robin Hanson  rhanson at gmu.edu  http://hanson.gmu.edu
Assistant Professor of Economics, George Mason University
MSN 1D3, Carow Hall, Fairfax VA 22030-4444
703-993-2326  FAX: 703-993-2323