[ExI] watson on jeopardy
lubkin at unreasonable.com
Thu Feb 17 16:26:49 UTC 2011
>Jeopardy? For each question there is only one
>right answer, and therefore only one right move
Aside, at least once a game there's a question
with more than one valid answer. (Sometimes it
didn't come up but I spotted it anyway.)
Contestants must proceed based on Trebek's
initial ruling. If the research team confirms
that the answer given was actually correct,
scores are adjusted. I have seen a contestant
brought back another day when they could have
plausibly won the game had their answer been deemed correct.
There's a similar flaw in many kinds of
test-taking, e.g., the Miller Analogies Test. A
is to B as C is to ___, (1) D (2) E (3) F (4) G.
E is the only answer accepted as correct. But
smart-you sees an interpretation whereby it's G.
What should be done is provide space with each
question to optionally provide a rationale. If
the expected answer is given, accept it. If a
different answer is given, see if there's a
rationale and it makes sense. (Still won't help
if the test scorer is a dolt who doesn't
understand your rationale, but it's an improvement.)
Otherwise both Jeopardy and the tests become
guessing games. Not what's the right answer, but
what's the one they would have thought of.
Easy to find on: LinkedIn Facebook Twitter Quora Orkut
More information about the extropy-chat