The Era of Immersive Interactions
June 2, 2015, Society for Information Display Symposium, San Jose, CA—Brian Krzanrich from Intel talked about the coming revolution in man-machine interfaces in the SID’s first keynote address. The era of immersive interactions will be as significant as any previous changes in electronic systems.
The industry and society are on the cusp of a major change in the way we interact with devise. The interfaces to machines has evolved from a 2-D surface to a more humanistic way to work with our devices. The first fifty years of Moore’s Law have allowed devices to change to be faster, smaller, and lower power. Following that trend has been the change in compute from patch cords, to punch cards, to keyboard and display, to graphical with a keyboard and mouse, and now moving towards more personal and connected.
The computer world has been moving towards smaller and more personal for a while, and the latest feature is connected as seen in the many mobile devices we carry. Even these personal devices—laptops, tablets, and smart phones—are giving way to the next generation of personal systems, wearable electronics. Along with the electronics and ICs, the displays have also evolved, from CRT, to LCD, to OLED, and others. Each generation has higher resolution and better performance metrics. The user interface has also changed, from command line, to GUI, and now to more human and personal, as shown in the movie “Her”.
The main problem is that the user interface continues to be fairly dumb and 2 dimensional. Touch interfaces are a change, but still use a 2-D paradigm. In a sense, the user interface is like a toddler, point at something and say “I want”. The way to change the interface is to give the computer the ability to see and hear like a person. Future computers will have eyes and ears to complement their knowledge of you and your preferences.
So far, the early stages of change are coming together. Speech recognition is getting to be comparable to a human listener for accuracy. The next step is to give the computer vision. Intel’s work in this area has resulted in our Real Sense technology, which uses an RGB camera, an infrared projector, and two IR cameras to provide a sense of depth space. The whole assembly fits on a small PCB that can fit into the bezel of a monitor or inside the case of a smart phone.
The unit provides a software view of the world and provides the capability of 22 tracking points. An SDK provides a starting point for new developments and includes motion tracking tools and some facial expression recognition capabilities. Among the capabilities is a chroma-key type background erasure function. The tools include a number of proven algorithms that can enable gesture control and other different user interfaces. The tools are available at www.Intel.com/realsense
Together the tools and sensor platform enable a new type of video experience that will be much more immersive, yet be more intuitive. The emerging ecosystem will facilitate new apps and will drive changes in operating systems to make them more user friendly. The common thread for all the changes will be due to the 3-D vision capabilities. The new capabilities will also enable industrial and commercial apps in areas like shipping, robotics controls like drones, and many others.
When the vision functions are coupled with appropriate software, the computer cam become a 3-d scanner and the resulting scans can be used as the models for 3-D prints. The computer’s ability to see will allow image creation that can be used within other programs, so you can put am avatar of yourself into the game.
The number of new use cases will grow as more tools become available, and as developers become more familiar with the new image functions. a quick app development allows multiple users to edit and collaborate in 3-D with the z-space software. The vision functions allow your hands and other tools to interact with 3-D models. When developers become comfortable with the concepts, we should see easier and more intuitive user interfaces, and the computer and the user can share the 3-D spaces.
Artificial reality displays in 3-D will change the human-machine and machine–human interactions. A demonstration game “understands” the normal physical boundaries of the world and can change actions within the space depending on movement mode of a person, car, or helicopter. The helicopter can move over objects and across boundaries while the ground units must go around the obstacles and have to stop at the edges of the table or will fall off.
The future of use interfaces and experiences will include projected images in the air. The illusion of an image floating in the air will create the potential for many new functions and apps. Wearables will combine augmented reality with 3-D vision. The software, systems integration, and other design requirements at OEM and other developer places will make more new APIs and refine the algorithms to create new markets.
