Disruptive Evolution of Computing
May 24, 2016, Society for Information Display symposium, San Francisco—Steven Bathiche from Microsoft talked about the ongoing changes in all facets of computing. The changes over time include processing power, form factors, and interactivity. The next generations of computing will continue the levels of changes.
Computers have evolved from room-sized behemoths requiring many kilowatts of power to machines requiring a large closet, to mini-computers, PCs, and now to a device that you can wear on your wrist. The functions of information display also have changed in size. The first systems used CRTs for basic interface and the displays grew larger.
At one point, flat panel displays emerged, offering new form factors and mobility to the computer user. While the screens continued to increase in size, the new technologies enabled more size diversity and also increased the specialization for any compute device. The electronics no longer dominated the size of the packaging, so compute systems could shrink while offering greater performance.
These new computer classes that cold combine mobility and computing changed the definitions of work and play. These new systems offer new capabilities and functionality, but didn't fully displace the previous generations of machines. The increased specialization for the new devices offer convenience at the cost of decreased functionality for some tasks.
The growing diversity of functionality and interactivity call for new types of display devices. One project developed a 40-inch horizontal display that can accommodate multiple users and input types. The touch-enabled screen has embedded imagers to detect touch functions, and the supporting software can detect gestures, read QR and other codes for enhanced reality functions. in effect, the integrated sensor and display allows the computer to see at least part of the world. This project show some hardware finds a unique niche.
The smaller Surface devices are similar to laptop computers and have a 3:2 aspect ratio that closely matches that of paper. The original intent was to make the tablet feel like paper and pen input. This differs from most other computers, which use mechanical devices for input. For most people, mechanical functions are left-brain centric while the right brain is the portion that is most involved in innovation and creativity. Studies show that students in college who use a keyboard for input retain less than their counterparts who use a pen and paper. The keyboard is less immersive and intuitive than a pen, due to the more mechanical nature of the keyboard.
One challenge for pen inputs is that an ideal tablet would have all of the characteristics of paper, no parallax or lag. Tests show that the position sensors need at least a 1 kHz (1 ms) refresh rate to give the impression of zero latency. The software has to distinguish between a palm print and a fingertip or stylus when detecting gestures so the base tablet system can be as intuitive as possible, meaning no menus are required.
Given these capabilities, now the system can offer other functions like enhanced penmanship on the fly, shadow drawings and auto completion of objects, and many other functions. The analog nature of the inputs also engenders new search capabilities based on the graphical characteristics of the inputs. The system offers something close to computational ink.
These large surface displays with integrated touch and image sensors can identify separate users and therefore enable 3-D aware tele-presence operation in a room-sized volume. This "magic window" can permit remote users to interact with the users at the touch screen further enhancing collaboration. The continuum of mobility and computing allows small computers and devices to drive larger ones, simultaneously enhancing and differentiating their diversity.
Achieving all of these capabilities is still a hard problem. The cursor tracking software functions have to interface to over 10 sensors and the display has to have wide-area 3-D capabilities approaching a holograph. While latency is a big issue, the integrated imagers can see 1-2 feet beyond the screen. The Kinect technologies underlie much of the imaging functionality, enhanced with high-fidelity eye tracking. The combination of 3-D display and cameras allows the computer to interact with the users' hands and also with more of their world.
New displays and new modes of interactivity increase the diversity of compute functions and lead to new applications. The ongoing evolution of computers calls not for more pixels, but for interactive, smart pixels to change and improve the relationship between users and computers.