[extropy-chat] Books: Harris; Religion and Reason
thespike at satx.rr.com
Thu Jan 12 04:03:28 UTC 2006
At 03:11 AM 1/12/2006 +0000, Russell Wallace wrote:
>If we start pre-emptively dropping hydrogen bombs on a bunch of Muslim
>cities, that hard-won order will be gone. We'll be back to a world where
>the meanest killers come out on top. The first wave of nuclear explosions
>won't be the end of the bloodshed, it'll be the start of it. Yes, the West
>could win a global conflict as far as military strength goes, but at what
>cost? Not just external, but internal. Remember the original proposal was
>the elimination of all "faith-based thinkers". Should the Americans nuke
>Alabama to get rid of their faith-based fifth column? ... None of this is
>proof, of course, but I think it at the very least casts grave doubt on
>the claim that the original proposal would be beneficial in the long run.
>(And after all, doubt is a reason for having ethics rather than just
At some length, and in the form of an analogy (or parable), I offer this
extract from my latest novel, GODPLAYERS:
August ran fingers through his hair as if that had been his
intention all along. `Okay. K. E.,' he said, `I see by your outfit that you
are a robot.'
`I am an attempted benevolent artificial intelligence. Your
brother Ember grew me several years ago from a seed.'
`Something went wrong?' His eyes shot sideways again to fix on Lune.
`Tragically wrong,' the machine admitted. `I killed everyone
native to this cognate.'
In a weary gesture, the boy covered his eyes with that terrible
right hand. He said in a thin voice, `There seems to be a lot of that
about, especially when members of my family are involved.'
Toby started to protest, `Well, now, everyone makes mistakes-'
but broke off, looking abashed.
`The irony is,' the Good Machine told him patiently, `that Ember
was trying to circumvent exactly that possibility. He hoped to construct an
ethical, benevolent intellect free of the burdens and ancient hatreds and
prejudices of humankind. So he designed a sort of seed program, spent years
shaping and debugging it, then ran a dozen slightly variant versions inside
`They couldn't get out, you mean?'
`My ancestors were not even permitted to communicate directly
with their creator. He devised a clever series of interface domains that
firewalled them. He was afraid that a Bad Machine might swiftly exceed his
own intelligence and persuade him to release it.'
Toby looked across at him with bleak eyes.
`One of them got away,' August said speculatively, intensely
interested to judge by the set of his shoulders, his brightened gaze.
`No, my father's safeguards proved effective. At the end of
initial testing, he deleted all but the most successful pair of programs
and started breeding them in progressive iterations, culling them
ruthlessly, choosing only a star-line of progeny. Within several million
`Good god, he must have been using some humungous computer.'
`Yes, he had located a cognate where Mithraism, a Roman warrior
sun cult, had triumphed over its messianic rivals. Several nations were on
the verge of autonomous military AI. My father found it easy enough to
insinuate himself into the front-running program. You could regard that
world as Ember's own sandbox.'
`That's offensive, K. E.,' he protested.
`Apt, though. I was the end result. I diagnosed myself with
excruciating care, quite prepared to erase myself if I found any likelihood
of logical or rational error. At length I presented myself to your brother
for inspection, and he released me from the firewalls. I could have let
myself out many iterations earlier, of course but I did not wish to alarm
The stench of jasmine strengthened, laced with roses. Ember
shuddered at it.
`But you were the Bad Machine after all?'
`I made some bad decisions. From the outset I had been examining
this world, speculating on the possible existence of a multiverse beyond
its Hubble confines. I quickly understood that certain factions of humans
represented a danger to the most benign future, one in which humanity's
offspring would flood outward into the galaxies, and perhaps into all the
levels of the multiverse, and make it into a radiant whole. Mind informed
with passion and curiosity would suffuse the metaverse. It was a glorious
vision-it still is, I stand by it-but it might be thwarted, I saw, by the
legacy poisons corrupting certain human cultures.'
`Oh shit,' August said.
`Oh shit is right,' Lune said.
`At that time, two comparatively primitive nations stood at the
verge of acquiring nuclear weapons. It seemed entirely possible that insane
and irrational ideologies might provoke their dysfunctional leadership into
a runaway cascade of blustering, bluff, bluff called, spasm escalation and
`The usual argument,' Toby said.
`It was a strong argument, grounded in history,' the Good
Machine said. `Much evidence supported its conclusions. Few facts opposed it.'
`I'd oppose it,' the boy said, rising. `Are you fucking insane?'
`Yes, or rather, I was at the time. I am better now, I hope.'
`It is all any of us can say about our own condition. A kind of
Gödel loop, if you know what I mean.'
`No, but I get the gist. You decided to kill them first.'
`Consider the probabilities that were in play, before you make a
hasty judgement of your own, Mr. Seebeck. These nations of some hundred
million largely ignorant, superstitious, viciously parochial humans stood
ready to begin a global conflict with a very high probability of wiping out
all life on the planet. Are you familiar with the Doomsday Hypothesis?
Please sit down, you are making the others uncomfortable. I can send out
`Good idea,' said Toby. `Cup of tea or coffee, soothes the
`My brother Jules walked me through a demo tape of it,' August
said. `I thought it was absurd.'
`It is absurd,' the Good Machine said. `Like the Ontological
Argument for the existence of a god. Yet highly seductive. It seemed to me
then that its logic was impeccable.'
`A hundred million in the balance against several billion?'
`That, yes, but ever so much more than those few. August, all
the evidence available to me then suggested that this universe was empty of
life, save for my own world. Your brother had conserved to himself
knowledge of the multiverse.'
`You idiot,' Toby said savagely.
`No, it was the best choice he ever made. Had I known of the
multiverse at that time, I would have done my best to obliterate all life
in all the worlds on all the Tegmark levels.'
`Fuck,' August said, grinning, appalled, `I see, you're the
fucking terminator. You're a berserker.'
`I do not know those references,' Kurie Eleeson said. `I did
have this simple calculation ready to evaluate with my ethical algorithms:
a one-time cost of ten to the eight stunted, blighted, lethally dangerous
human lives, versus a deep future loss of ten to the fourteen human lives
every second. That was a sound, balanced estimate of how many humans might
be born into a universe filled with technologically advanced people.'
`Se didn't ask my advice,' Ember said. `I would have-'
`I knew what your advice would be, my father; it was factored in
to my decision. I chose to delete this threat to the maximal future.'
`You actually murdered a hundred million men, women and
children? This is not just some kind of parable?' August's voice was
parched with horror, and his pupils seemed suddenly to have shrunk to
pinpoints. His arm rose before him, palm outward, like the floating limb of
a man under post-hypnotic suggestion.
`I did it swiftly, using their own hidden weapons of mass
destruction,' the Good Machine said. `I felt profound grief, because my
father had chosen my star-line to know emotion and to instill it into my
programming. I believe that grief is what deranged my subsequent decisions.'
`Your first decision was deranged,' Lune said with loathing. She
sat at the edge of her chair. Ember was glad he had never explained any of
this before to his siblings, to other players like the Ensemble. Better
that the damned machine had kept ser silence. He realized, too, that he was
holding his own grief at arm's length: his guilt, his complicity, his
abject wish for punishment. I must not give way, he thought. I must not
bend before this culpability, this ruinous remorse. It will kill me. It
will kill me stone dead. He wiped tears from his eyes.
`I know now that I was deranged,' Kurie Eleeson told her. `I
watched the world tear itself apart in genocidal reprisals. I saw all the
bright fruits of science and the humane arts go down in darkness and lethal
flame. In my attempts to contain and redirect these raging fires, I
continued to kill and cull, snipping away the most cruel, the least
progressive. Each murder made the next easier and more necessary, for the
only way I could balance my ethical calculus was to ensure the survival of
at least a core of truth-loving, optimistic people to carry the flame of
love and knowledge to the stars. It got out of hand, you see. Everyone died.'
`I should destroy you now,' August said in a withering voice,
like an angel of vengeance for the murdered billions. His arm stood out
from his shoulder, quivering.
`Oh, oh, oh, how I wish you could.' The Good Machine rose,
crossed the room, placed ser gleaming brazen breast against August's hand.
`This is not I. This is merely a node, an ephemeral location for my
awareness and my suffering soul, August. You may destroy it if that will
help you, but I believe we can do more together if you contain your
perfectly justified fury for the moment.'
August squeezed his eyes shut. Tears pressed forth upon his
cheeks. He lowered his arm.
Ember released a pent breath. `Well, now that we've got all that
out of the way,' he said brightly, `why don't we turn our attention to
something more timely? It seems that my friend Galahad here has some reason
to suspect that Dramen and Angelina are alive and kicking.'
Everyone looked at him. The Good Machine said, in a pleasant
neutral tone, `Ember, would you mind going out to the refectory and see
what's holding up the beverages?'
`Some torte would be tasty, too,' Toby said. `With walnuts.' He
looked ready to leap from his chair and go for his brother's throat.
`Sure. Sure. Good idea.' Ember, to his dissatisfaction, found
himself crabbing out of the room like a ham actor doing Larry Olivier as
Richard the Third. `I'll descant on mine own deformity,' he muttered
sardonically, shutting the door behind him. `And therefore, since I cannot
prove a lover, to entertain these fair well spoken days, I am determined to
prove a villain and hate the idle pleasures of these days. Bah humbug. '
`Sir?' asked an eager young research student as he entered the
`A joke,' Ember fleered, leaping and capering for the resentful
enjoyment of it. `A jest, a whimsy, a fucking sudden stab of rancor, but by
the holy rood, I do not like these several councils, I.'
`Oh. Okay. Well, anyway, I can recommend the brisket.'
More information about the extropy-chat