How Would IBM’s Quiz-Show Computer, Watson, Do as a Competitor in the National Science Bowl?

Enlarge Photo

Photo courtesy of NSB

Dr. Eric Brown, IBM Watson Health, explains how the IBM computer system, Watson, played the quiz show Jeopardy! and the challenges of natural language processing.

"This is one of the smartest audiences to whom I have ever presented," said Eric Brown, after a question-and-answer session with the hundreds of students gathered for the plenary lecture at the 2016 U. S. Department of Energy's National Science Bowl^® Science Day.

Brown, who is the Director of Watson Algorithms at IBM Watson Health, described the Grand Challenges program at IBM, designing and preparing IBM's Watson computer system that played and won the Jeopardy! quiz show in 2011, and the quest for solutions in natural language processing (NLP).

He explained that Grand Challenges is a label used broadly to describe IBM's deliberate approach to generating innovation. "IBM looks at areas where there's an enormous gap in current capability and uses that as a challenge," Brown said. "IBM is an industrial research lab – dedicated to innovate, invent, and push the state of art of technology. We do basic research and have a sense of market and client needs and applications and are able to translate that into real products and solutions that can benefit Society in general." To be chosen as an IBM Grand Challenge, the problem must be so enticing that it supports a larger team to go at the challenge and eliminate other problems along the way, with more time and investment to produce something more powerful and successful. "You see that if you achieve that goal," said Brown, "it will have an enormous impact."

Brown offered an example. In 2004, America followed Ken Jennings on the TV quiz show Jeopardy! as he racked up an historic 74 game-winning streak. Research directors at IBM watched and wondered if they could build a computer able to play Jeopardy!. "We did a scientific feasibility study – we looked at the game, our current technology, measured the gap, and convinced ourselves it was achievable," recalled Brown. It was a very public demonstration of scientists and engineers working to solve a hard problem – natural language processing (NLP) - and produce meaningful experimental results: scientific achievements and a public demonstration event.

To prepare for the televised match of man and machine, IBM engineers and computer scientists advanced the technology, built the Watson computer system, loaded its memory, fashioned a solenoid to mechanically press the buzzer, ran the system through 8,000 experiments to test it and gather data on its performance, then announced they were ready.

The machine had been designed and tested to play in a game scenario. It needed to strategize – make good selections on the board and place wagers in daily doubles and Final Jeopardy!. It needed to be fast: able to understand the question, search for probable answers in its vast content repository, judge the evidence, then reply. IBM's interest was not in how well Watson heard the clue; the interest was how well Watson understood the clue and came up with the correct answer.

The game was set up so that Watson got the clue in ASCII as soon as the clue was displayed on the board. In the two televised rounds, Watson won, scoring $35,734 and $77,147 and taking the $1 million prize.

Watson's win showed the machine's ability to understand complex language – question responses required high precision. You can use search engines to find a document, but you have to read the document and pull out the best answer.

There are untold ways to express an idea and human language is grounded in human cognition; we work on understanding and using language from our birth. So how do you give a machine this ability? When computers understand natural speech patterns, it means easier access, improved search results with more accurate retrievals, and faster input of data and text. Brown's current research focus is natural language processing in health care.

Electronic medical records is an area of great need for technological improvements. The effort, Brown explained, is not to add tools and interfaces to steer humans; the effort is to make the computer smarter to use the language and data the humans naturally produce. Heath care is an unstructured data problem; there are text, figures, images, video and sound to be analyzed, saved, and later retrieved. To effectively employ medical knowledge today requires use of Big Data; there are millions of articles to store, index, access, and retrieve. And more medical data is created every day.

But for each patient, Brown told the students, it is Big Data "in the small." A patient's electronic medical record might contain just a few megabytes and it is unstructured because, traditionally, the MD's medical notes are unstructured. The physician's workflow is: preparation (a two-minute review of the patient's history), take the current history and do the physical, access the history again, assess to diagnose (connect the problem to the related information and relevant medications and lab results) and, finally, plan. How can a computer help a physician perform these tasks in a timely and accurate manner? The computer must be able to use NLP to search medical records, understand the different measurements, terms, synonyms, and negative matches. It must differentiate between ‘cold – an illness caused by a virus' and ‘cold – the sensation of not being warm' and ‘cold – the body temperature measures less than 98.6.'

Brown sees the foundational challenge of improving natural language use as "When we interpret language, we use context."

He offered two pieces of advice to the high school audience: "A Public Service Announcement – anything you put on the internet – Snapchat, Facebook photos – stays out there. Employers might look at that," he said. "And employers who understand Big Data can find it."

He also suggested to bright students who contemplate taking less than four years for college that they might reconsider. "Early graduation seems efficient and cost-effective but you miss out on those outlier classes that develop out-of-the box thinking skills and insights," he told them. "Don't rush things. Take every semester available to you."

When asked if Watson would play the National Science Bowl^®, Brown replied "No."

The Office of Science is the single largest supporter of basic energy research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information please visit http://science.energy.gov.

Sandra Allen McLean is a Communications Specialist in the Office of Science, Sandra.mclean@science.doe.gov.