[extropy-chat] No rejection of science! Re: SI morality
Eliezer S. Yudkowsky
sentience at pobox.com
Fri Apr 30 16:05:29 UTC 2004
Hal Finney wrote:
> I think my entire post can go through unchanged if you simply substitute
> "the practice of science" or "the scientific consensus" for the shorthand
> "science" everywhere.
These are extraordinarily different things.
The practice of science is a social process.
The consensus of science is an opinion poll.
The actual working part of science is Bayesian probability theory, which
individual scientists and their social dynamics partially and imperfectly
> It is this approach, of taking scientific things one at a time, which
> I think is dangerous. You are setting your judgement above that
> of the scientific community. It is dangerous to try to cherry-pick
> among scientific issues to decide for which ones we will agree with
> the consensus and for which ones we will decide that the scientific
> consensus is wrong. This makes it too easy for our individual lapses
> in rationality to find an excuse to enter and influence our thinking.
Science intrinsically requires individual researchers setting their
judgment above that of the scientific community. The social process of
science encourages people to do the work and recognizes when they have
done the work. The social process is not an actual human brain, has not
the power of intelligence. If individuals do not have novel opinions and,
yes, disagreements, for the scientific process to recognize as correct,
there is no science.
We might not know that Einstein was right about relativity if there were
no scientific process to recognize him. But Einstein would still be the
one who thought through to the answer. Yes, Einstein relied on a social
process of science to recognize Maxwell, but it is still humans who power
The overall rationality of academia is simply not good enough to handle
some necessary problems, as the case of Drexler illustrates. Individual
humans routinely do better than the academic consensus.
> Essentially I am advocating the idea of following the scientific consensus
> faithfully; you might even say, blindly. The reason is because our
> errors of rationality are so pervasive and seductive that we are more
> likely to be wrong than is the scientific consensus.
> This is not an easy principle to follow! It feels like an abdication of
> responsibility, like an abandonment of critical thinking. But when I look
> within, these feelings do not come from the part of me that loves truth,
> they come from the part of me that loves myself. They are a manifestation
> of ego. They come from an emotional desire to be the master of my fate,
> which means making my own decisions about what to believe and what not
> to believe. Delegating these matters to any outside social institution,
> even one whose track record in approaching the truth is greater than
> anything mankind has ever developed, goes against powerful mental
> instincts. Nevertheless I claim that this is what we have to try to do.
Yes, the Way of rationality is difficult to follow. As illustrated by the
difficulty that academia encounters in following. The social process of
science has too many known flaws for me to accept it as my upper bound.
Academia is simply not that impressive, and is routinely beaten by
individual scientists who learn to examine the evidence supporting the
consensus, apply simple filters to distinguish conclusive experimental
support from herd behavior. Robyn Dawes is among the scientists who have
helped document the pervasiveness of plausible-sounding consensuses that
directly contradict the available experimental evidence. Richard Feynman
correctly dismissed psychoanalysis, despite the consensus, because he
looked and lo, there was no supporting evidence whatsoever. Feynman tells
of how embarassing lessons taught him to do this on individual issues of
physics as well, look up the original experiments and make sure the
consensus was well-supported.
Given the lessons of history, you should sit up and pay attention if Chris
Phoenix says that distinguished but elderly scientists are making blanket
pronunciations of impossibility *without doing any math*, and without
paying any attention to the math, in a case where math has been done. If
you advocate a blanket acceptance of consensus so blind that I cannot even
apply this simple filter - I'm sorry, I just can't see it. It seems I
must accept the sky is green, if Richard Smalley says so.
I can do better than that, and so can you.
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
More information about the extropy-chat