[extropy-chat] SI morality

[Hal Finney]

Chris Phoenix writes:
> I've gone through a crisis of faith with regard to scientists recently. 
> In many areas, I've come to realize, scientists are far too 
> self-assured.  They think they're practicing science, when in fact they 
> are merely contributing to science.  A lone scientist can run 
> experiments, observe, make hypotheses, form opinions... but cannot fully 
> practice science, because science can only emerge from interactive 
> criticism.  We are all too fallible to trust ourselves to generate good 
> science without lots of help.
> ...
> So how can science be reported to the real world?  If one scientist's 
> opinion isn't trustworthy, what about lots of opinions together? 
> Michael Crichton has called this "consensus science," and correctly 
> attacked it.  It's no more than a popularity contest for ideas, and the 
> popularity of an idea has little to do with its truth.

This is a dangerous road to take.  I'd be concerned that if I started
off doubting the practice of science as a guide to truth, I might as
well send in for my membership card in the Flat Earth Society, because
that's where I'd end up.

You complain above that individually, scientists can't practice science
because that requires interactive criticism.  But in fact, most individual
scientists do work in a framework of interaction.  Most scientists
that I've known are actually very cautious about criticism, and do
their utmost to make their presentations and publications bulletproof.
They'll go out of their way to mention any weaknesses or ambiguities
in their theories specifically in order to pre-empt their critics from
raising those points.  Science is a sport where defense counts more than
offense, from my observations.

And then you go on and criticize consensus science as being no more
than a popularity contest.  But this again overlooks the tremendous
importance of criticism in the scientific process.  A scientifically
unsound theory, even if popular, cannot withstand criticism for long.
There is too much temptation to jump onto the critical side once people
see that it is going to win.  Science rewards successful critics,
and this self correcting mechanism is part of what has made science so
successful as an institution.

The real problem with abandoning science is that you will have no guide
to truth in our complex world.  No one can become familiar with all of
the technical details relevant to the issues we face.  By abandoning
science you are explicitly turning away from the people who have spent
their entire lives acquiring expertise in these areas.

Do you really think you are better able to weigh the many complexities
around, say, global warming than those who have devoted their careers to
studying the atmosphere and climate?  Or similarly with other questions
like the safety of genetically engineered plants?  Or even, yes, the
feasibility of nanotech?

I have an extreme belief in the importance of being open minded.
I've written at length about the dangers I see in ideologies, the way
they blind us and control our thoughts.  I've been strongly influenced by
the results that Robin Hanson has reported and extended about the ways
we fool ourselves, how we don't really seek the truth even though we
think we do.  I believe in seeking the truth.  To the extent that I have
an ideology, that's it.  I try to look for those mechanisms in my mind
that are operating to push me off the path to truth, and to compensate
for them as well as I can.

One of the principles I follow is that if I believe something that
mainstream science disagrees with, I am probably wrong.  It's for the
reasons given above.  I'm not smarter than those guys, at least not
the smartest ones of them.  And their expertise in these areas is far
deeper than my own.  Plus they have this incredibly complex and elaborate
process of modelling and testing and subjecting each others results to
intense criticism, while my uninformed notions on those topics undergo
no such rigorous trials.

The lesson I learned from Robin is that if I disagree with someone
else, it's an accident of history which position I ended up with.
I could have just as easily been in his shoes.  Hence I should have
no presumption that I am probably right, when there is a disagreement.
Given this perspective, when I am going up against a scientific consensus,
the odds are overwhelming that the scientists are right and I am wrong.

It looks to me like these attitudes are the only appropriate ones to adopt
for someone who sincerely seeks the truth.  We have to try to discard
or at least overcome our prejudices and egotistical belief in personal
correctness and superiority.  We have to be willing to change our minds
when we come up against a situation where the experts disagree with us.

Without the guidance of the best advice and analysis available on a
subject, I would be concerned about being vulnerable to all kinds of
quackery and fraud.  We have many crazy beliefs right on this list.
Some here refuse to accept the reality of global warming.  Some believe in
psychic powers.  Some reject the link between HIV and AIDS.  Some believe
the universe is packed full of intelligent life.  Some believe that Israel
caused the 9/11 attacks.  Some believe in cold fusion.  And that's not
even mentioning the whole complex of beliefs about the Singularity.

Rejecting science means rejecting the best and most successful institution
mankind has ever developed for finding out the truth about the world.
It puts you onto a dangerous path fraught with tempting falsehoods that
can lead you astray.  As I suggested above, you better set aside money
for your membership in the Crackpot League, because that's where this
road ends.

Hal Finney