Natural Sensing and Interface Technologies
August 5, 2014, Emerging Display Technologies Conference, Santa Clara, CA—Achin Bhowmik from Intel presented the company’s vision for new displays and interfaces. The ongoing developments in sensors and displays will forever alter the way people interact with computers. The interactive displays will become truly 2-way visual devices into the world.
Most early interfaces to objects have been through touch. Nicholas Tesla described a control interface at the turn of the last century. In the ’50’s the first TV remotes appeared, as wired controllers. Next developments led to wireless remotes with ultrasound or light as the transfer medium. One interesting aspect of these technology changes is that the advertisers had to reduce the length of ads, since it became easy to change channels with a remote.
Following the early developments of small computers, Xerox developed the mouse, and others created the first touch screens about the same time. These interfaces still use touch, because the hand and especially the fingertips are some of the most sensitive and fine motion areas on a body.
Now, we are getting numerous sensors that will allow us to map functions to distinct areas, just as our brains have areas of specialization. The brain uses most of its processing power for visual and audio inputs to provide more environmental cues. Multi-modal sensors can provide a bio-mimicry and more natural and immersive interfaces.
Perceptual computing can add sensors and actuators to mirror the human senses and provide interfaces that use more than touch. Use of the other senses and inputs from the cloud and environment can infer the user’s context and emotional state for a natural, intuitive, and immersive interface.
To make the computer even more like a human, it needs binocular visual sensing, rather than the very limited 2-D currently available. 3-D imaging for sensing will have one sensor for color and other ways to recognize depth. A 3-D image sensor would allow full-range gestures beyond tap, swipe, and pinch. This “real sense” technology is possible with an IR projector, IR sensor, and a color sensor, roughly similar to the Microsoft Kinect.
Intel has created two versions of this interface, one for short range sensing for computers and another for ranges up to 5M for TV controls. The sensor is factory calibrated in a static mode, and is dynamically recalibrated on power up. Currently, resolution is +/- 1 percent of the distance, growing at greater distances. The vision system is currently on a PCB about the size of a pencil, but could be reconfigured for wearable devices.
To encourage people to develop apps for this sensor array, they have SDK to enable 3-D video control functions. The library functions include detection capabilities and a template for user interfaces. This type of interface will be much better than a remote control, and will allow immersive collaboration in AR and VR activities. The additional hardware will not materially increase the cost of materials for a laptop computer.
The inclusion of 3-D capabilities also enable automatic measurements for scanning objects for 3-D printing or other tools that need a fully scanned image. The base components for interactive displays are already here. The improvements in sensors and computer processing power will make better user interfaces.
