Interactivity Solutions
August 5, 2014, Emerging Display Technology Conference, San Jose, CA—The last session at this conference considered interactivity. Calvin Hsieh from NPD Display Search opened the session, followed by Brian Daly from Synaptics, David Bordui from Cima NanoTech, and Russell Jordan from AMS.
Hsieh noted that smart devices have different user interfaces that change with the device. Some companies, like Apple, have a consistent interface across all devices, while other companies don’t do as well. The adding sensors to the basic package helps with orientation and location, but Siri cannot read the sensors.
As a result, most smart devices require voice or gesture inputs to get some interaction, like “OK Google”. Natural language is still far off due to the accuracy requirements for critical apps. Within the mobile ecosystems, the tablets and phone stand alone, while wearables are attached to the user. The user interface has to change because of this difference in proximity and the size of the input device.
The large (over 10 inch) touch panels have nothing to drive greater demand. The ASPs are falling lading to reduced investments in new technologies. The tablet market is starting to consolidate as the various devices start to cannibalize the next levels. There is little difference between a tablet and a notebook with a detached keyboard. The touch market is changing technologies to in cell or on cell, but really has no place to go.
The next frontier for user interfaces will be in gestures and voice inputs. Touch is OK for mobile devices, but is not well suited for fixed devices. A new user interface in needed for the large number of fixed in place devices like TVs and desktop monitors. TVs are starting trials with voice inputs, gestures, and track pads. The graphical user interface has to use large icons since the user is far from the display. The icon size limits the TV apps in quantity and in capabilities.
Game consoles are moving to more optical interfaces. The PS3 started an optical sensor and the PS4 changed to stereo vision. The Kinect 2 uses a time-of-flight active light source with structured light to generate distance information for the console. These advancements are moving from games to other areas. Project Tango is developing robotic eyes for devices and the machine vision uses structured light for depth resolution.
Developments in artificial reality find that gestures are promising, but he area still needs algorithm refinements for higher resolution and accuracy. The user interface revolution benefits from the improvements in computers and will eventually migrate to perceptual computing. The move to better user interfaces depends upon more sensors to increase usefulness and trust.
Daly talked about adding intelligence to the display to better integrate the touch and display functions. The move to thinner and lighter displays is the driver for changes in touch technologies, to in- or on-cell sensors. Now, touch is becoming a differentiator wit proximity sensors to dim the backlight when the phone is near your head. Other changes include waterproof devices and new functions like wake-up, pair, proximity, and glove and fingernail sensing.
Integrated multi-function sensors can simplify the supply chain as the displays move to in- or on-cell touch technologies. On-cell sensors are moving from a 2 layer layout with crossovers to isolate the transmit and receive signals, to a single layer with special patterning. The underlying technology uses a hybrid transmit signal on the TFT layer, and the next generation will put both transmit and receive circuitry on the TFT layer.
Bordui described touch user interfaces on large format displays and other surfaces. Normal touch technologies do not work well with the thick glass on ATMs or other industrial-grade surfaces. Instead, new substrates like polycarbonate can have touch sensors molded into the surface.
The wet coating applies a self-assembling material with a 3-5 micron cross section. On a film, the sheet resistance is under 20 ?/?, and can be processed to be less than 1 ohm for antenna or EMI shielding. The low resistance leads to faster scn rates for better interactivity or lower power.
Jordan advocates IR gestures for interfaces. In normal communications, we use all five senses plus perception, whereas with machines, we mainly use touch. Sight and sound are coming as the number and types of sensors increase. The new senses enable machines to have new user interfaces that are more natural.
IR sensors facilitate 2-D and 3-D cameras. Alternatives include ultrasound and passive IR. Theses sensors can be used in combination with touch to change the user interface. Active IR gesture control requires an ambient light sensor and a color sensor to detect movements and distance.
One use for the active IR is Mobeam, a barcode transmitter for mobile devices that sends the underlying barcode to the reader, which usually cannot read the barcode on the device due to the surface reflections. The technology enables gesture control of a device at a distance which is good for large screens. The keys to greater adoption of active IR are performance, size, a good and simple supply chain, and proven technologies.
Developers need to consider the whole ecosystem when making new user interfaces as the available technologies in IR, capacitance, gesture, etc. grow with advances in technologies. Most of these interfaces are in smart phones, but are moving to other platforms over time. The challenges for developers are in the software and algorithms, sensor technologies, prior user experience, and the perception of usefulness.
