Automotive AR Displays

March 25, 2014, GPU technology conference, San Jose, CA—Kobi Ben Zvi from Mishor talked about heads-up displays and augmented reality for cars. There is an ongoing trend to include more augmented reality functions in cars to provide greater ease of use, safety, and greater access to relevant information.

The advance of augmented reality in cars is an expansion of the Internet of Things. Currently about 15 billion devices have machine to machine to communications enabled, as is projected to grow to over 50 billion devices in the next five years. The question for all is how does the user benefit from the addition of augmented reality to his normal environment.

Vision dominates human functions. We use vision for perception, learning, cognition, and many other functions. Augmented reality enhances these functions by adding contextual information to objects that we perceive. Putting these technologies into cars can help reduce the 5-1/2 billion hours per year that drivers spend in traffic in the United States. In addition, traditional directions from GPS units only provide minimal information in getting from point A to point B, while AR can provide a contextual display to indicate information about locations between the endpoints.

Currently, the main concept is to put an overlay on a 3-D display. The challenges are in the real world the driver does not just look at the 8 degree angle field of view directly ahead. The roads have curves, changes in elevation, and the visual system is subject to motion and shaking transmitted from the car. Current heads-up displays only cover a small area on the windshield.

For this technology to take off, it needs to have a robust, and common user interface as well as good designs for human and machine interactions. Research can help to determine suitable icons and data structures for a universal user interface. Other work is needed to provide displays that cover the full width of the windshield.

The displays will need to provide dynamic information in a manner that is not confusing or distracting to the driver. Changing arrows and expanding dots are a couple of possible images for nonlinear directions. Having the display extend across the entire windshield means that directional information is always in the line of sight of the driver, even when the drivers not looking straight ahead.

Ongoing research is being undertaken in Germany and in Michigan as well as within other research organizations and academic institutions. One possible implementation is to eliminate the camera and navigate through a cloud based mapping and location system like Google. One research topic that is getting good results is a lane departure warning from a video input. This safety function already exists in fairly rudimentary form in some of the high-end cars.

Another challenge is clear to locate the images, and to keep them from being obscured when they show up on other vehicles. Most heads-up displays only provide information for the straight ahead view, which places the information on the back of the preceding vehicle.

Augmented reality is starting to become integrated into the infotainment center, but it will take at least 3 to 5 years for large-scale deployment. One implementation issue is whether the AR functions use additional hardware, or are part of the existing HUD hardware. The other concern is whether the information is self-contained or part of a connected infotainment system.