[extropy-chat] On difficult choices (was: Books: Harris; Religion and Reason)

[Eliezer S. Yudkowsky]

Jef Allbright wrote:
> 
> Sometimes a military leader is faced with the difficult choice of
> sacrificing some of his troops in order to save the rest.  Sometimes
> an individual will sacrifice himself to allow others to survive in an
> overloaded lifeboat.  Sometimes a surgeon will advise a patient to
> undergo radical amputation in order to have a chance at life. 
> Sometimes a politician will risk loss of popularity in order to
> contribute to a greater good.

Sometimes people can be hypnotized by difficult choices.  One recalls 
Elrond, in Tolkien's prehistory to _The Lord of the Rings_, pleading 
with Isildur to throw the Ring into Mount Doom.  In the movie version we 
get to see this (and as far as I know, it's faithful to Tolkien): 
Elrond and Isildur actually standing at the Crack of Doom, Isildur 
holding up the Ring, and then...

Elrond:   Throw in the Ring!
Isildur:  Nah.
Elrond:   Okay.

So what should Elrond have done?  Push Isildur screaming into the Crack 
of Doom?  A fine deed that would have been, to set to the credit of the 
Ring...  So Elrond let Isildur go, resulting in some untold number of 
casualties in the War of the Ring a few centuries later.

Should we blame Elrond for that?  Well, if it was me, I sure would blame 
myself.  Just because I have ethics doesn't mean I'm not responsible for 
their consequences.

Plus the Ring killed Isildur anyway.
And Isildur was lucky.  He could have ended up as Gollum.

Elrond had plenty of options besides pushing Isildur into Mount Doom. 
He could have bopped Isildur on the head and then used his sword to 
nudge the Ring off the edge.  Worst case scenario, Elrond bops Isildur 
on the head, calls in his lieutenants, strips off his own armor, and 
*volunteers* to be pushed into Mount Doom if he can't manage to nudge 
off the Ring, throw off the Ring, or step off the edge.  If Elrond 
wasn't willing to sacrifice himself, he was *obligated* to call for 
volunteers, and if that made him feel awful that was *his* problem.

Elrond was so focused on the obvious wrong way to solve the problem that 
he didn't see the creative right ways.  His great failure wasn't that he 
lacked ethics, it was that he didn't know how to use them.  He thought 
his ethics were supposed to be heroic disadvantages.  If Elrond had just 
taken for *granted* that he couldn't push Isildur off the edge, instead 
of agonizing, he would have seen easier and better solutions.

It won't always be that way.  We don't live in so kind a universe.  But 
for Elrond it was so, even without Tolkien intending it.

Did anyone else notice this, when they read the book, or watched the movie?

The theory behind the Singularity Institute is that it's possible to 
*save the entire damn world* without killing people, pointing guns at 
people, telling people what to do, or any of the usual bullying 
tribal-chief solutions that instantly pop into people's heads when they 
consider political problems.  That's not idealism, it's intelligence. 
History teaches us that the "difficult" choices, the obvious wrong ways 
to solve the problem, DON'T FRICKIN' WORK.  Stalin broke plenty of eggs, 
but where are the omelets?

So don't make excuses in advance for ethical failures.  People are so 
hypnotized by "difficult" choices that they don't look *hard* for a 
creative solution.  They just go straight off and make the "difficult" 
choice.  Taking the "difficult" option is not difficult, it's easy and 
convenient.  That's why people spend so much time looking for excuses to 
do things the "difficult" way.

So what's really difficult?  Thinking.  It can be frickin' hard to think 
of a good solution, you've got to, like, actually sit down and 
concentrate.  And sometimes, yes, it's painful and inconvenient - for 
*yourself*, not some convenient outside victim who has to be 
"sacrificed" - to do things the right way.  It's not always easy.  So 
don't make your excuses in advance, or you'll shoot yourself down before 
you start.

-- 
Eliezer S. Yudkowsky                          http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence