[extropy-chat] Books: Harris; Religion and Reason

Damien Broderick thespike at satx.rr.com
Thu Jan 12 04:03:28 UTC 2006

At 03:11 AM 1/12/2006 +0000, Russell Wallace wrote:

>If we start pre-emptively dropping hydrogen bombs on a bunch of Muslim 
>cities, that hard-won order will be gone. We'll be back to a world where 
>the meanest killers come out on top. The first wave of nuclear explosions 
>won't be the end of the bloodshed, it'll be the start of it. Yes, the West 
>could win a global conflict as far as military strength goes, but at what 
>cost? Not just external, but internal. Remember the original proposal was 
>the elimination of all "faith-based thinkers". Should the Americans nuke 
>Alabama to get rid of their faith-based fifth column? ... None of this is 
>proof, of course, but I think it at the very least casts grave doubt on 
>the claim that the original proposal would be beneficial in the long run. 
>(And after all, doubt is a reason for having ethics rather than just 
>utilitarian analysis.)

At some length, and in the form of an analogy (or parable), I offer this 
extract from my latest novel, GODPLAYERS:


           August ran fingers through his hair as if that had been his 
intention all along. `Okay. K. E.,' he said, `I see by your outfit that you 
are a robot.'
           `I am an attempted benevolent artificial intelligence. Your 
brother Ember grew me several years ago from a seed.'
           `Something went wrong?' His eyes shot sideways again to fix on Lune.
           `Tragically wrong,' the machine admitted. `I killed everyone 
native to this cognate.'
           In a weary gesture, the boy covered his eyes with that terrible 
right hand. He said in a thin voice, `There seems to be a lot of that 
about, especially when members of my family are involved.'
           Toby started to protest, `Well, now, everyone makes mistakes-­' 
but broke off, looking abashed.
           `The irony is,' the Good Machine told him patiently, `that Ember 
was trying to circumvent exactly that possibility. He hoped to construct an 
ethical, benevolent intellect free of the burdens and ancient hatreds and 
prejudices of humankind. So he designed a sort of seed program, spent years 
shaping and debugging it, then ran a dozen slightly variant versions inside 
           `They couldn't get out, you mean?'
           `My ancestors were not even permitted to communicate directly 
with their creator. He devised a clever series of interface domains that 
firewalled them. He was afraid that a Bad Machine might swiftly exceed his 
own intelligence and persuade him to release it.'
           Toby looked across at him with bleak eyes.
           `One of them got away,' August said speculatively, intensely 
interested to judge by the set of his shoulders, his brightened gaze.
           `No, my father's safeguards proved effective. At the end of 
initial testing, he deleted all but the most successful pair of programs 
and started breeding them in progressive iterations, culling them 
ruthlessly, choosing only a star-line of progeny. Within several million 
           `Good god, he must have been using some humungous computer.'
           `Yes, he had located a cognate where Mithraism, a Roman warrior 
sun cult, had triumphed over its messianic rivals. Several nations were on 
the verge of autonomous military AI. My father found it easy enough to 
insinuate himself into the front-running program. You could regard that 
world as Ember's own sandbox.'
           `That's offensive, K. E.,' he protested.
           `Apt, though. I was the end result. I diagnosed myself with 
excruciating care, quite prepared to erase myself if I found any likelihood 
of logical or rational error. At length I presented myself to your brother 
for inspection, and he released me from the firewalls. I could have let 
myself out many iterations earlier, of course but I did not wish to alarm 
my parent.'
           The stench of jasmine strengthened, laced with roses. Ember 
shuddered at it.
           `But you were the Bad Machine after all?'
           `I made some bad decisions. From the outset I had been examining 
this world, speculating on the possible existence of a multiverse beyond 
its Hubble confines. I quickly understood that certain factions of humans 
represented a danger to the most benign future, one in which humanity's 
offspring would flood outward into the galaxies, and perhaps into all the 
levels of the multiverse, and make it into a radiant whole. Mind informed 
with passion and curiosity would suffuse the metaverse. It was a glorious 
vision-­it still is, I stand by it-­but it might be thwarted, I saw, by the 
legacy poisons corrupting certain human cultures.'
           `Oh shit,' August said.
           `Oh shit is right,' Lune said.
           `At that time, two comparatively primitive nations stood at the 
verge of acquiring nuclear weapons. It seemed entirely possible that insane 
and irrational ideologies might provoke their dysfunctional leadership into 
a runaway cascade of blustering, bluff, bluff called, spasm escalation and 
global Doomsday.'
           `The usual argument,' Toby said.
           `It was a strong argument, grounded in history,' the Good 
Machine said. `Much evidence supported its conclusions. Few facts opposed it.'
           `I'd oppose it,' the boy said, rising. `Are you fucking insane?'
           `Yes, or rather, I was at the time. I am better now, I hope.'
           `You hope?'
           `It is all any of us can say about our own condition. A kind of 
Gödel loop, if you know what I mean.'
           `No, but I get the gist. You decided to kill them first.'
           `Consider the probabilities that were in play, before you make a 
hasty judgement of your own, Mr. Seebeck. These nations of some hundred 
million largely ignorant, superstitious, viciously parochial humans stood 
ready to begin a global conflict with a very high probability of wiping out 
all life on the planet. Are you familiar with the Doomsday Hypothesis? 
Please sit down, you are making the others uncomfortable. I can send out 
for refreshments.'
           `Good idea,' said Toby. `Cup of tea or coffee, soothes the 
savage breast.'
           `My brother Jules walked me through a demo tape of it,' August 
said. `I thought it was absurd.'
           `It is absurd,' the Good Machine said. `Like the Ontological 
Argument for the existence of a god. Yet highly seductive. It seemed to me 
then that its logic was impeccable.'
           `A hundred million in the balance against several billion?'
           `That, yes, but ever so much more than those few. August, all 
the evidence available to me then suggested that this universe was empty of 
life, save for my own world. Your brother had conserved to himself 
knowledge of the multiverse.'
           `You idiot,' Toby said savagely.
           `No, it was the best choice he ever made. Had I known of the 
multiverse at that time, I would have done my best to obliterate all life 
in all the worlds on all the Tegmark levels.'
           `Fuck,' August said, grinning, appalled, `I see, you're the 
fucking terminator. You're a berserker.'
           `I do not know those references,' Kurie Eleeson said. `I did 
have this simple calculation ready to evaluate with my ethical algorithms: 
a one-time cost of ten to the eight stunted, blighted, lethally dangerous 
human lives, versus a deep future loss of ten to the fourteen human lives 
every second. That was a sound, balanced estimate of how many humans might 
be born into a universe filled with technologically advanced people.'
           `Se didn't ask my advice,' Ember said. `I would have-­'
           `I knew what your advice would be, my father; it was factored in 
to my decision. I chose to delete this threat to the maximal future.'
           `You actually murdered a hundred million men, women and 
children? This is not just some kind of parable?' August's voice was 
parched with horror, and his pupils seemed suddenly to have shrunk to 
pinpoints. His arm rose before him, palm outward, like the floating limb of 
a man under post-hypnotic suggestion.
           `I did it swiftly, using their own hidden weapons of mass 
destruction,' the Good Machine said. `I felt profound grief, because my 
father had chosen my star-line to know emotion and to instill it into my 
programming. I believe that grief is what deranged my subsequent decisions.'
           `Your first decision was deranged,' Lune said with loathing. She 
sat at the edge of her chair. Ember was glad he had never explained any of 
this before to his siblings, to other players like the Ensemble. Better 
that the damned machine had kept ser silence. He realized, too, that he was 
holding his own grief at arm's length: his guilt, his complicity, his 
abject wish for punishment. I must not give way, he thought. I must not 
bend before this culpability, this ruinous remorse. It will kill me. It 
will kill me stone dead. He wiped tears from his eyes.
           `I know now that I was deranged,' Kurie Eleeson told her. `I 
watched the world tear itself apart in genocidal reprisals. I saw all the 
bright fruits of science and the humane arts go down in darkness and lethal 
flame. In my attempts to contain and redirect these raging fires, I 
continued to kill and cull, snipping away the most cruel, the least 
progressive. Each murder made the next easier and more necessary, for the 
only way I could balance my ethical calculus was to ensure the survival of 
at least a core of truth-loving, optimistic people to carry the flame of 
love and knowledge to the stars. It got out of hand, you see. Everyone died.'
           `I should destroy you now,' August said in a withering voice, 
like an angel of vengeance for the murdered billions. His arm stood out 
from his shoulder, quivering.
           `Oh, oh, oh, how I wish you could.' The Good Machine rose, 
crossed the room, placed ser gleaming brazen breast against August's hand. 
`This is not I. This is merely a node, an ephemeral location for my 
awareness and my suffering soul, August. You may destroy it if that will 
help you, but I believe we can do more together if you contain your 
perfectly justified fury for the moment.'
           August squeezed his eyes shut. Tears pressed forth upon his 
cheeks. He lowered his arm.
           Ember released a pent breath. `Well, now that we've got all that 
out of the way,' he said brightly, `why don't we turn our attention to 
something more timely? It seems that my friend Galahad here has some reason 
to suspect that Dramen and Angelina are alive and kicking.'
           Everyone looked at him. The Good Machine said, in a pleasant 
neutral tone, `Ember, would you mind going out to the refectory and see 
what's holding up the beverages?'
           `Some torte would be tasty, too,' Toby said. `With walnuts.' He 
looked ready to leap from his chair and go for his brother's throat.
           `Sure. Sure. Good idea.' Ember, to his dissatisfaction, found 
himself crabbing out of the room like a ham actor doing Larry Olivier as 
Richard the Third. `I'll descant on mine own deformity,' he muttered 
sardonically, shutting the door behind him. `And therefore, since I cannot 
prove a lover, to entertain these fair well spoken days, I am determined to 
prove a villain and hate the idle pleasures of these days. Bah humbug. '
           `Sir?' asked an eager young research student as he entered the 
           `A joke,' Ember fleered, leaping and capering for the resentful 
enjoyment of it. `A jest, a whimsy, a fucking sudden stab of rancor, but by 
the holy rood, I do not like these several councils, I.'
           `Oh. Okay. Well, anyway, I can recommend the brisket.'


Damien Broderick 

More information about the extropy-chat mailing list