Watson in the Cloud
May 9, 2012, All About the Cloud Conference, San Francisco—Stephen Gold from IBM described next generation computers based upon recently demonstrated capabilities, such as those in the Watson system. The new systems will be instrumented, interconnected, and intelligent and will enable new ways to use data.
Today, most data is unstructured. Users need to take that unstructured information and use it to improve business outcomes. Information is the new oil, only when it is used is it valuable. As a result, the new systems for handling those data need to be learning systems that can adapt the changes in data quality.
This new era of computing is a result of the ongoing evolution of data processing. The first systems were just for tabulation, to help the US census. The next systems that came were programmable computers that operated on algorithms. Now we are in an era where computer systems can actually learn on their own to address some of the bigger problems.
A learning system will change the user emphasis from search to discovery. For example, today a Google search is deterministic, but is not context aware. Content that is probabilistically determined can consider the context of the query and provide a more focused response. Search looks for records, while discovery finds sources of engagement.
IBM attempts a grand challenge technology demonstration about once a decade. In’06, some engineers noticed that Ken Jennings was accumulating a long string of victories on Jeopardy. They thought is would be interesting to make a computer system that could challenge the best trivia experts in the show.
After some years of development, they became a contestant on Jeopardy and trounced the two human participants. The challenges for the computer are in working in natural language input, creating search parameters, applying probabilities to the responses, and selecting the best answer, all within the 3 seconds allowed for a buzz-in and before the human contestants.
Now, the technology is being applied to less trivial areas. The first commercial application for the technology was in health care last August. The second was in financial services, announced in March. Other applications areas are in the works.
The drivers for the development are that data volumes are growing exponentially. Over 90 percent of all data in existence has been generated in the past 2 years. Of that data, 80 percent is unstructured, and only about 20 percent of all data available is useable in the enterprise.
The Watson technology provides a analytic solution for the data volume. Watson is comprised of a system of systems, and has integrated 41 subsystems. These systems are designed to understand natural language, generate and evaluate hypotheses, and learn and adapt to changing requirements.
The system understands people. In the context of health care, medical practitioners have to deal with the data explosion. Every five years, the data needed to practice doubles. One in five diagnoses is inaccurate or incorrect, leading to 1.5M errors in the US every year. 81 percent of all doctors spend less than 5 hours a month reading the latest journals.
The medical application uses a technique called differential diagnosis to help the doctor determine possible and most likely health issues. It then gathers additional data on the patient and iterates its diagnosis. The system can then suggest follow up queries and suggest the most relevant tests for the various conditions.
At Wellpoint, the nation’s largest insurer, they are using Watson to evaluate preauthorization for treatments. The system evaluates the requested tests and uses some efficacy measurements to approve or disapprove those tests. They loaded 25K cased into the system for learning, including outcomes data and are about to put the systems online for the doctors.
The system is stateful – meaning that it remembers previous information—iterative, and capable of massive data ingest. The next trial will be at Memorial-Sloan Kettering cancer institute. They have 20M patient records and will help to build out an app for cancer testing and treatment.
In the financial services area, they are working with Citibank to develop a consumer-facing app to provide guidance to investors. Citi has lots of information on various investment vehicles and have found that most investors are overwhelmed. By putting the content into a useable context, they can offer focused offerings to their consumer and institutional investors. This repackaging of information enables the investors to make more informed decisions about risk-reward and helps the user be less passive about their money.
Watson is comprised of a number of platforms. The main functions are data, analytic, and Watson engine. IBM is in the process of releasing an API for the Watson engine to facilitate new apps. The system will be offered as a hybrid cloud service
IBM is planning to become the content curator and in the process create more training sources. The cloud structure permits the capabilities of the various engines to be available to many users. The cloud permits flexible consumption, dynamic capacity, hybrid delivery, and a short time to value.
The new computer system will increase usability through its natural-language input systems. It is helping business get ready for the massive data management tasks that are upon us now and growing to unimaginable sizes. The inclusion of smart analytics allows users to align-anticipate-and act upon all the data in their environment, while the learning capabilities ensure that the datasets have the greatest utility.
