Watson and Jeopardy

My roommates and I usually catch Jeopardy at least once a week, so we were of course excited to watch Watson, the IBM machine, take on Ken Jennings and Brad Rutter in Jeopardy. So far, Watson's done extremely well, getting 13 straight answers right last night, and finishing way ahead of the competition at the end of the game yesterday. In particular I was intrigued by a few things:
  • In case you haven't watched football with me, I generally annoy everyone else in the room the entire game by urging the team with the ball to be more aggressive: pass more, go for it on 4th down, and stop kicking field goals. The data backs me up. Similarly when I watch Jeopardy, I get upset when players land on the Daily Double - the one chance they have to win a ton of money very quickly - and bet very conservatively, throwing away their chance to leave their competition in the dust. So I was intrigued by this comment from Ken Jennings on Watson's performance:
    Daily Doubles aren't distributed randomly...basically, it had thousands of old Jeopardy games in its head and knew where to look. And got very lucky. Most players go top-down, but some hunt for Daily Doubles like Watson. Brad Rutter calls this an arms race: if one player does it, you have to join in to keep up.
    Clearly Watson, as a computer that can compute expected value and forecast his expected winning percentage, has figured out that the Daily Doubles are valuable - he routinely picks questions in the $800 and $1000 row to start the game, and bets big - $6000 at one point - on the question. This play is forcing the other players in the game to improve their strategies. Hopefully this trend will extend to normal games of Jeopardy, as it increases the variance and makes the game more fun to watch. However, Jeopardy's normally a one-off game so it's possible player strategy won't improve.
  • Lots of people make the mistake of assuming a person is "smart" because they are good at Jeopardy. I don't think Jeopardy is a very good measure of intelligence; it measures your knowledge of obscure trivia, as well as your ability to recall it, but there's more to intelligence than an encyclopedia. It's possible this skill is correlated with intelligence (maybe intelligent people read widely, or have better recall), but Jeopardy skill doesn't prove intelligence per se. It's good to see people realizing Watson knows a lot but isn't "smart" - it can't recognize when another contestant gave the same answer, for example.
  • Watson's buzzer pressing mechanism is lightning fast, so it wins the buzzer on every question where it computes the answer by the time Trebek finishes reading the question. I thought this was unfair, but then realized putting an artificial damper on Watson's reaction time would be stupid - if it's a better Jeopardy player under the current rules, then great, and if it's unfair, they should change the rules so everyone gets an equal shot at the question, or use some other mechanism to award first crack at an answer.

Liked what you read? I am available for hire.

2 thoughts on “Watson and Jeopardy

  1. Andy McKenzie

    Interesting post! This: “Hopefully this trend will extend to normal games of Jeopardy, as it increases the variance and makes the game more fun to watch. However, Jeopardy’s normally a one-off game so it’s possible player strategy won’t improve.”

    .. I don’t think it’s true. Excitement is game is based on the number of times that your expectation of who is going to win changes, and how wildly that is. Those daily double with huge risks will lead to really exciting moments, but will actually make the rest of the game _less_ exciting as there will be a lower expectation of lead changes.

    Doesn’t mean they shouldn’t do it, of course.

    They should change the system so humans change have their brains read by some sort of EEG, and train it to know when they want to “press the button” to buzz in. The EEG reads those signals and calls in when you want to buzz in. This makes it more fair… our synapses versus machine transistors.

  2. kevin

    I would also venture that Jeopardy rewards fast buzzer pressing more than knowing the answer, and the 2nd and third place contestants know just as many answers as the 1st place person, they’re just slower on the buzzer. To the audience though it looks like the first place contestant is way smarter than the others. Ultimately if people realized how much of the game depends on button pressing speed I think they would be fairly dissatisfied with it. EEG’s might help but you’re still in a situation where 3 people are competing to buzz in first to answer a question where they all know the answer.


Leave a Reply

Your email address will not be published. Required fields are marked *

Comments are heavily moderated.