For Watson, Jeopardy! is elementary
IBM researcher explained Watson’s magic at MIT on Monday
Quick! Take a clue, and break it down into keywords, finding out what it’s asking for. Rack your brain and generate a list of hundreds of possible answers to said question. Then evaluate each answer produced for validity, using varying types of evidence. Figure out the best possible choice and report it. And do it in two to six seconds. Such a process is routine for supercomputer Watson, explained David Gondek, an IBM researcher who spoke at MIT on Monday, just before Watson began its highly anticipated competition against human Jeopardy! champions Brad Rutter and Ken Jennings.
(Watson, in case you haven’t heard, won the competition handily.)
Gondek talked about his experience at IBM developing Watson, as well as the inner workings of the machine, to a packed audience in 3-270. Several teams of researchers at IBM spent four years working on Watson. MIT Computer Science and Artificial Intelligence Laboratory research scientist Boris Katz also aided the project.
Watson, which runs on 2,800 IBM POWER7 processing cores, employs machine learning techniques and runs several algorithms in parallel to produce fast answers to questions, explained Gondek.
According to Gondek, Watson does not use simple keyword searches in its question-answering routines because they are inefficient. Instead, Watson beings by analyzing the Jeopardy! clue to figure out what “type” of answer is needed, whether it be a baseball player or an ancient civilization. To aid in this analysis, developers gave Watson so-called “semantic frames” in the form of a “subject — verb — predicate,” so that it knew, for example, that “inventors patent inventions,” and “authors write books,” Gondek said.
Next, Watson searches its massive framework of content — it does not connect to the Internet and is completely self-contained — and generates a list of possible answers. Each answer is then evaluated, as Watson tries to find various types of evidence for support. Gondek explained that such evidence can come from many categories, including spatial, temporal, and taxonomic clues. After all of this, each possible answer is ranked. The one that has the highest “confidence” rating is the machine’s answer.
Watson is both “blind and deaf,” said Gondek. It read clues electronically at the same time that Jeopardy! host Alex Trebek read the questions out loud. After the clue is read, an indicator light on set cuts on, signaling the opportunity to buzz in. At the same time, Watson is allowed to press its own buzzer, which is mechanically controlled — Gondek described it as a “very fast solenoid,” to the laughter of Course VI undergraduates in attendance.
When deciding whether or not to buzz in, Watson incorporates information about its confidence into its answer, as well as the particular game situation. Potential wagers made by the machine also depend on several factors. If Watson owns a large lead, for example, it may only bet a couple hundred dollars on a “Double Jeopardy!” question.
According to Gondek, Watson was trained with over 100,000 sample Jeopardy! questions. In addition to being able to track Watson’s progress, researchers were also able to access problem areas and “teach” Watson what to do with certain clues.
Watson wasn’t perfect. On the television screen, producers placed a graphic showing Watson’s top three answer choices, along with the confidence it had in each one. These provided insight into its thought process and showed what at times appeared to be nonsensical answers. For instance, for the “Final Jeopardy!” question on Tuesday, the clue asked for a certain U.S. City. Watson’s answer? “Toronto.” Gondek said that, unfortunately, Watson is too “complex” to know its complete thought process for every answer.
In the future, IBM plans to utilize the technology of Watson in other areas, such as business or health care, where Watson could help in diagnosing illnesses, said IBM Program Director of Technical Recruiting William R. Strachan PhD ’67. The system behind Watson — the POWER7 cores — are commercially available.