[extropy-chat] Re: Overconfidence and meta-rationality

[Hal Finney]

Russell Wallace writes:
> I agree completely; Eliezer and I did this some months ago, for
> example, regarding the feasibility of hard takeoff. Now we're both
> smart people who've studied the issue carefully, and come to different
> conclusions, so what was the rational thing for us to do? Answer: each
> of us argued our case until we boiled it down to flat difference of
> intuitions, whereupon we nodded, "Now I see why you believe what you
> do - go ahead and prove me wrong."

So did you "agree to disagree"?

Doesn't it bother you that a smart and knowledgeable person like Eliezer
has come to a different view of the facts of the matter?  You're both
born into the same world, products of the same evolutionary process;
you're exposed to different information, and ultimately your estimations
of probabilities are based on these causal factors.  Do you accept that
if you had been exposed to the experiences Elizer had, you would have
come up with his estimation of the probabilities, rather than yours?
If so, then doesn't that tell you that there are facts in the world,
information you could have gained, which would make you share his beliefs?
And isn't knowing that such persuasive information exists, even without
direct access to the information itself, enough to make you doubt your
position and consider it equally likely that Eliezer's view is correct?

Well, I realize that this is a pretty long chain of inferences.  I'd be
curious to hear where you diverge from this argument for why you should
not be able to knowingly disagree with someone whom you view as rational
and honest, and who you think believes the same of you.

Hal