[extropy-chat] Scientific standards of evidence

[Max More]

Chris -- given your comments, you'll find the following two papers very 
interesting. Here are my reviews of them. If you go to the URLs, you'll 
find links to other relevant material.

I, too, am concerned about the integrity of the scientific process as it 
currently exists. Doing something about it is just about my top priority 
for ExI, if we had the resources.

Max


Management Science: What Does it Have to do with Management or Science?
Marketing Bulletin by J. Scott Armstrong
http://www.manyworlds.com/index.asp?from=CO&coid=CO10270312354673

Some people believe that personal experience is the gold standard of 
knowledge. Dismissing theory in favor of intuition and experience can be a 
fatal mistake because human cognition and perception are rife with errors, 
biases, distortions, and limitations. At the same time, management theory 
often fails to qualify as "management science", typically systematizing 
wobbly assumptions into a superficially compelling form. Scott Armstrong 
investigates what real management science can tell us about the usefulness 
of formal planning, portfolio matrices for decision making, mission 
statements, the pursuit of market share, preparing sales forecasts, survey 
design, and predicting the outcome of a conflict situation.

Scientific management research uses objective, replicable procedures to 
compare various approaches, method, or theories. Only by adhering to 
standards of scientific rigor can management theory produce results worth 
paying attention to. Armstrong shows that this research can and has 
produced useful results but also that most management literature is 
useless. More worrying is that the proportion of useful work is declining. 
The useful results that are published are typically unseen, rejected, or 
ignored. Armstrong does an excellent job of explaining the reasons for this 
situation as well as suggesting ways to improve the communication of 
important findings in management science.

After analyzing the shortcomings of the research methodology, publication, 
and dissemination processes, Armstrong looks at the possibilities. He 
recommends taking all research findings and putting them on the Internet, 
subjecting them to continuing peer review and avoiding the distorting 
dynamics of the journal process. In addition, he suggests creating 
management science "impact sites" to focus on objectives that are relevant, 
explicit, measurable, and challenging. Impact sites could be used to 
compare universities and their departments in terms of the usefulness of 
their research.

Armstrong provides specific and helpful recommendations to university 
deans, reviewers, authors, and practitioners. The contribution of the 
Internet could be powerfully augmented by using expert systems and 
software. Expert systems can incorporate new procedures and software new 
findings, making the default choice to stay current while preventing 
knowledge leakage. Far-sighted companies might even contribute to the 
process of improving management science by supporting nonprofits who work 
on these projects. Armstrong has also should that nonprofits produce more 
useful results but lack motivation to publish. By contrast, universities 
have resources but are removed from real problems and businesses are near 
the problem but usually lack the resources for doing the research.



Reaping Benefits from Management Research: Lessons from the Forecasting 
Principles Project
by Ruth A. Pagell; J. Scott Armstrong
The Wharton School, University of Pennsylvania
http://www.manyworlds.com/index.asp?from=CO&coid=CO10270312354673




_______________________________________________________
Max More, Ph.D.
max at maxmore.com or more at extropy.org
http://www.maxmore.com
Strategic Philosopher
Chairman, Extropy Institute. http://www.extropy.org <more at extropy.org>
_______________________________________________________