[extropy-chat] silent night

[Harvey Newstrom]

On Dec 22, 2004, at 7:21 PM, Robin Hanson wrote:

> On 12/22/2004, Harvey Newstrom wrote:
> Did you actually go look at the IEM web site and the many papers they  
> have there?  If so, you might have found:  
> http://www.biz.uiowa.edu/iem/archive/forecasting.pdf

This paper claims that existing evidence only shows the Iowa markets  
predicting elections 24 hours before the election.  They claim that no  
evidence has been seen or analysis performed on longer-term  
predictions.  I find it hard to believe that in the history of that  
market, nobody tried to analyze the data.  More likely, I believe that  
like me, nobody was able to extract evidence of long-term predicitons.   
This seems to support my previous research and discount years of  
previous claims made by people for the Iowa market.

Given such an admission, I had (have?) high hopes that this new paper  
would give such evidence.  However, it has been in "working draft"  
stage for two and a half years, and was never actually completed,  
peer-reviewed and finally published.  If tis paper is providing the  
proof, why has it never been finished?  If it is accurate that no  
previous analysis exists, and this "first" analysis has been left  
unfinished after two and a half years, I am left with the idea that no  
final analysis showing good results actually exists.  I am skeptical  
that such evidence will be presented anytime soon.

A single unpublished, unpeer-reviewed, and seemingly  
abandoned/unfinished paper does not carry a lot of weight with me.   
Especially when this paper itself confirms that no other evidence  
exists besides this paper.

The data presented in this paper shows very small sample sizes and very  
large variability with no consistency between elections.  The paper  
itself says that there is no way to calculate margin for error, so it  
is not clear if the sample size is large enough (which I doubt) or the  
variability/randomness is small enough (which I doubt).

For example, the look at presidential elections only shows four  
elections.  One was less accurate than polls most of the time.  One  
consistently grew less accurate over time as the election approached.   
That's two out of the four elections that don't look good to me.

I also disagree with the measurement methodology in some places.  They  
take the last market price at the end of the last day of any polling  
period, while the polls data are averaged over those days.  Shouldn't  
the same measuring method be used for both?  Shouldn't the market data  
be averaged over the polling period also?  Otherwise, we aren't just  
comparing polls to this market, but comparing earlier days polls with  
later market values.  This doesn't seem consistent or fair.

Also, the data only shows whether the market "beats" the polls, but  
doesn't show how much.  I think being wildly wrong should weight more  
than being a fraction of a percent right, but the calculations aren't  
made this way.  They also seem to chose arbitrary dates to look at.   
Why not choose evenly spaced dates or look at the continuous graphs?   
The graphs don't look as good as the snapshots do.  The market is often  
wrong for weeks or months at a time.  It also flip-flops in its  
predictions on many of the graphs.  These almost daily errors and  
reversals are not reflected in the snapshots data.

I will admit that I did not do math like this in my analysis.  I looked  
at their historical graphs online.  They show the markets flip-flopping  
back and forth, and long periods of being wrong.  Even if they  
statistically turn out to be more right than wrong at the end, it is  
wildly inconsistent.  There is no point at which we can look at a  
future election and calculate who the winner should be.  The  
calculation changes almost daily.  The final time of convergence to the  
correct answer ranged from weeks to days and in some graphs never  
occurs until the final day.  This seems to make predictability  
difficult.  Even if the math works out (which has never been published  
in final form or peer-reviewed), it still looks like it will be a  
"significant" few percent more accurate, but not enough to make it a  
prediction tool.  If it predicted the 2000 election by 49.9 to 50.1  
instead 49.8 to 50.2, that doesn't give enough of an edge to predict  
presidential elections in advance.

> Did you look at any of the papers mentioning such claims and look up  
> their citations?
> For example, re an extropian angle you might have found:   
> http://hanson.gmu.edu/moretrue.pdf

Nice paper *about* predictions, but little evidence of historical  
success or specific methods.

> You are making this stuff up.  You apparently have no idea what  
> investors are doing.  Try:
>
> Barbara Kiviat,  
> <http://www.time.com/time/insidebiz/article/0,9171,1101040712 
> -660965,00.html>The End Of Management?, Time, Inside Business, A4,  
> July 12, 2004. http://hanson.gmu.edu/PAM/press/Time-7-12-04.htm

Let me be clearer.  I am not saying that "nobody" is doing this.  I am  
saying that "virtually nobody" is doing this.  A few minor  
counter-examples can't change the fact that 99% of investors have never  
heard of this stuff.  99% of the markets do not include idea futures.   
This is not the mainstream market of today.  The stuff being referenced  
here is more of a proof-of-concept demonstration, not real mainstream  
markets based on these ideas.

--
Harvey Newstrom <HarveyNewstrom.com>
CISSP, ISSAP, ISSMP, CISA, CISM, IAM, IBMCP, GSEC