[ExI] Watson On Jeopardy

[Richard Loosemore]

David Lubkin wrote:
> What I'm curious about is to what extent Watson learns from his mistakes.
> Not by his programmers adding a new trigger pattern or tweaking
> parameters, but by learning processes within Watson.
> 
> Most? successful people and organizations view their mistakes as a
> tremendous opportunity for improvement. After several off-by-one
> errors in my code, I realize I am prone to those errors, and specially
> check for them. When I see repeated misunderstanding of the referent
> of pronouns, I add the practice of pausing a conversation to clarify who
> "they" refers to where it's ambiguous.
> 
> Limited to Jeopardy, it isn't always clear what kind of question a
> category calls for. Champion players will immediately discern why a
> question was ruled wrong and adapt their game on the fly.
> 
> Parenthetically, there is a divide in competitions between playing the
> game and playing your opponent. Take chess. Some champions
> make the objectively best move. Emanuel Lasker chose "lesser"
> moves that he calculated would succeed against *that* player.
> Criticized for it, he'd point out that he won the game, didn't he?
> 
> I wonder how often contestants deliberately don't press their buzzer
> because they assess that one of their opponents will think they know
> the answer but will get it wrong.
> 
> Tie game. $1200 clue. I buzz, get it right, $1200. I wait, Spike buzzes,
> gets it right, I'm down $1200. Spike buzzes, gets it wrong, I answer,
> I'm up $2400. No one buzzes, I've lost a chance to be up $1200.
> 
> I suspect that it doesn't happen very often because of the pressure of
> the moment. (I know contestants but asking them wouldn't answer
> the question.) If so, that's another way for Watson to have an edge.
> 
> (Except that last night showed that Watson doesn't yet know what
> the other players' answers were. Watson 2.0 would listen to the game.
> Build a profile of each player. Which questions they buzzed on, how
> long it took, how long it took after buzzing for them to speak their
> answer, voice-stress analysis of how confident they sounded, how
> correct the answer was. (Essentially part of what an expert poker
> player does.)
> 
> I also wonder about the psychological elements. Some players
> seem to dominate a Jeopardy game. If you were playing Ken
> Jennings in his 63rd game, or a single game opponent who's up
> by $15,000, would you play better than you otherwise would or
> worse? (The initial strong lead that Watson had could have
> intimidated lesser adversaries.)

This is *way* beyond anything that Watson is doing.

What it does, essentially, is this:

It analyzes (*) a vast collection of writings.  It records every content 
word that it sees, and measures how "near" that word is to others in 
each sentence ... i.e. how many other words come in between.  It then 
adjusts these "nearness" measures as time goes on, to get averages.

So if its very first text input is "Mary had a little lamb" it would 
record "Mary" and "lamb", and give them a distance of 4.

If it then saw "Mary Queen of Scots" it would record a distance of 3 
between "Mary" and "Scot", and it would increase the distance between 
"Mary" and "lamb", because "lamb" was not in the second sentence.

And on and on and on.  Through billions of pages of text.  It would then 
have a table with one column for every word in the language and one row 
for every word, and each entry is the average "distance" between the words.

Then, when given a Jeopardy problem, it looks for answer words (or 
possibly phrases?) that are very near to the content words in the given 
sentence.  Then it forms a question with that word or phrase as the 
object, and its done.

Hence:  "What food do grasshoppers eat?"  Answer: "Kosher", because the 
most frequent places where "food" and "grasshopper" were mentioned in 
all those billions of input texts, were in places discussing the fact 
that grasshoppers are a food that is kosher.

Apart from various bits of peripheral processing to catch easy cases, to 
look for little tricks, and to eliminate useless non-content words, etc 
etc., that is all it does.

It is a brick-stupid cluster analysis program.

So, does Watson think about what the other contestants might be doing? 
Err, that would be "What is 'you have got to be joking'?"



Richard Loosemore


P.S.

(*) I am inferring the algorithm based on reports coming out of there, 
and the way it makes mistakes.  I havenot seen the code, obviously.