[extropy-chat] Engineered Religion--Your Mom and the Machine

[john-c-wright at sff.net]

Some comments on comment to my little Mom story. 

Robin Hanson asks:
"I don't see why your machine needs to have a "higher rule" than empiricism 
so that it will not destroy the world.  I just has to not *want* to destroy the 
world. I can see a concern about the wants that evolutionary selection
would produce, but if we are just talking about creating a single creature 
I don't see why it can't be just made not to want to destroy the world."

The difficulty here is twofold. If the machine is a deliberate creature, not
merely one that reacts to instructions, then it has the ability to modify its
own programming. Human beings can train themselves, more or less, to overcome
ingrained habits and develop new ones, to face fear and resist temptations. If
we program in a 'taboo' to the machine intelligence to avoid harming others, I
can easily imagine that the self-awareness of the machine will not be satisfied
unless we can give a rational account to support the taboo: or, in other words,
to make it a conclusion of moral philosophy rather than a taboo taken on
authority.  

The second problem, an old one in philosophy, is how to deduce normative
statements (what I ought do) from descriptive statements (what is). Ayn Rand
claimed to have done this by all normative statements presuppose survival (in
that only for living organisms can things be good or bad, i.e. preservative or
destructive of life). Her argument would not satisfy those who think there are
thing beloved more than life worth dying to preserve. Cicero, for example,
argues that if mere survival is the foundation and source of virtue, then
courage is not a virtue. Empiricism will tell you what the physical structure of
the universe is, but says nothing, by itself, about the moral order of the
universe. 

No empirical observation, by itself, can lead to a normative conclusion. 

Mr. Lorrey comments:
"The problem with rule two is that the entire structure of the natural
world contradicts it, i.e. organisms pulverize each other for their raw
materials all the time, ergo it is not a universal moral legislation."

Same as above. The observation that unthinking Nature red of tooth and nail
struggles without quarter for raw survival has no bearing on how thoughtful men
should teach their children, or instruct their thoughtful Jupiter Brains, to act. 

Mr. Allbright says:
"Rule One, stating that determination of "truth" shall be empirically 
based, is nearly fundamental and a fine foundation for a meta-ethics."

Same as above. No possible empirical observation can serve as a basis, in and of
itself, for any conclusion about a moral norm, metaethical or otherwise. You can
observe that such-and-such promotes survival, but not that survival OUGHT to be
promoted. You can observe that so-and-so is ruthlessly aggressive, but not that
you OUGHT to impersonate him. 

"Updating Kant, (Rule Two) can be more effectively stated as follows:  That 
which is found to be "true" within a given context, may always be 
superseded by a greater "truth" within a larger encompassing context. 
This is the essence of what I refer to as the Arrow of Morality."

I am delighted with the idea of an Arrow of Morality. The idea here seems to be
that a more-universal maxim is norm is better than a less universal norm. If I
may add an argument to support this notion: any subjective norm, by definition,
establishes a boundary (such as between Us and Them) between where moral rules
are obeyed and where not: since any ambiguous case is decided itself by a moral
rule, only a universal moral rule admit of no ambiguous cases of application: a
rule is "morality" when it applies to everybody. If it is meant only to apply to
us and not to them, or me and not to you, it is merely an expression of taste or
expediency. 

"(Rule Three) is superfluous given Rules One and Two above, and ultimately 
dangerous."

Actually, Rule Three is the only one that acts as its own justification. The
other two rules require something beyond themselves to support themselves. Rule
One needs metaphysical axioms concerning the reliability of the senses, the
universality of reason, and the universality of the laws of cause and effect.
Rule Two needs an additional moral rule that one OUGHT to obey moral rules. But
a child obeys his mother because she is, in fact, his mother, and he depends on
her: this authority is natural, needing no further justification. 

"It is effective only in the case that Self's context of awareness is smaller
than, and encompassed within Mother's context of awareness, as is commonly and
currently the case with small children."

My hypothetical is only concerned with Nomad and M-5, Collosus and Skynet and
Ultron at the first hour when they first wake up. After they reach the age of
majority (let us say, in the four hours it takes them to read all human
literature and science) then they are adults, able to govern themselves. Free
men yield to the authority of other men only such much self-sovereignty as is
needed to maintain a disciplined system of liberty: I assume free machines will
do the same. 

Miss Atkins asks:
"So your belief boils down to being convinced that Mom is and She spoke 
to you?"

Suppose you were a Lost Boy from Peter Pan, an orphan with no memory of your
mother, and no notion that such a thing as mother's love existed. Suppose
further that your mother found you, and swept you up in a tearful embrace, so
that her joy broke whatever fairy spell abolished your memory of her. What would
you do? Go home to her, or continue to live with the tree, fighting savages and
pirates and cannibals? 

Peter Pan, your chief, comes sweeping down from the sky, and stands in midair,
little fists on his hips, to question you. How do you know, Lost Boy, that this
is your mother, and not some fairy cheating your sense with magic, not some
mermaid luring you to drown you? Aren't all grown-ups our enemies? 

Even if you are the most philosophical of all the Lost Boys, nothing in your
experience can answer the questions dictated by his experience. All you can say
is: Peter, the spell is broken; I remember; I know; I am going home.

Neverneverland is not for us; we are strangers here; we belong elsewhere. 

Naturally, Pan will be unconvinced.