Augmented and Virtual Reality

May 2, 2013, Neuro-Gaming Conference, San Francisco—Augmented and virtual reality technologies are entering games. A panel moderated by Jason Jerald from nextGen Interactions considered the issues related to game design and development. panel members were Walter Greenleaf from the Stanford Longevity Institute, Laurent Scallie from Atlantis Cyberspace, and Palmer Luckey from OculusVR.

Scalie opened that VR systems are complex, with a high potential for failure. They incorporate many sensors, like gyros on your head and in your weapon. Now the equipment is going wireless and adding optical tracking. Even as the first games come out, the need for better middleware and more refined heads-up displays and tracking technologies. The games need to work better to suspend disbelief and provide a track to reality. There are diminishing returns on visual improvements since visual immersion and natural reactions leading to greater emotional gains are hard. The underlying hardware and engineers are getting better.
Luckey noted that technology advances enable improvements. VR headsets with head tracking and better displays are becoming available. Motion tracking is getting less expensive.
Greenleaf commented that after 30 years of research, neuroscience and VR are approaching an inflection point in technology and acceptance. So far, users have been slow to adopt the technologies. The combination of visual and audio produces greater immersion and the improving game engine performance is leading to more natural interactions. The changes in technologies and mass market availability are making the devices more affordable.
Outside of gaming, medical apps are providing the greatest impact. From surgeon training to treatment of mental health issues like PTSD, behavioral modification, and phobias, the VR technologies are helping people. One challenge is that the medical industry is very conservative and is just starting to adopt game and computer technologies.

Challenges?
Scallie offered that in the ’90’s, everything was proprietary, platforms and algorithms. Now the difficulties are economic, the field needs content. Another problem is motion sickness with a wide field of view. Cutting the cord will reduce friction and offer a more natural interaction with the environment. The industry needs fine-grain hand motion sensing.
Greenleaf considered that the barriers are not technical, but economic and regulatory. The VR industry needs more entrepreneurs. The basic technologies have few barriers left, the algorithms and engines are available. The gaps are in everyday activities like hugs and crying.
Luckey agreed that the technology issues are not the main barrier to adoption. No economic incentives are available. User acceptance is an issue since this new operating mode represents a change in the user experience. Gaming has been a social activity, but VR is only really available as a single person mode, resulting in lower emotional content.

Parallels to the ’90’s and lessons learned?
Greenleaf noted that the main difference is that the 90’s happened. There was lots of hype which won’t repeat. The culture is different.
Scallie suggested that the progress in technology offers a different landscape. The ecosystems exist—content creation, render, hardware platforms, etc. and now there are commercial products and standards.
Luckey said that technologies keep improving and costs drop. 3-D high frame rate and high dynamic range are possible.

VR help the senior community?
Greenleaf responded that moving VR into the senior citizen market is a great opportunity. The elderly are disconnected and the boomers are technology ambivalent, but connected. There is a big economic opportunity in senior health.
Scallie added that low friction and ease of use are important.
Luckey suggested that the boomer generation is low technology adoption. They use tools like Skype and others, but don’t interact with the data. The natural interfaces in VR allow better interactions versus the mouse. As an example, the Wii with a simple interface is much more popular than the PS in elder populations.

Minimum latency?
Luckey commented that sensor fusion is an issue, especially for stereo 3-D HFR. The phones are not capable yet, but mobile hardware will get there in 5 years. Less than 20ms latency can be improved with predictive tracking. One issue is the power for render.

Autism?
Greenleaf mentioned that VR is good for teaching social skills etc. but the patient still needs a partner.

Emotions and techniques?
Greenleaf offered that workarounds are possible, and evolving facial recognition software can detect micro-expressions. Users need stress inoculation and training. Getting to body language and facial expressions is hard.
Luckey captured videos and mapped them to game characters. This facet is over exaggerated in VR, so the industry needs to move to more natural views.

Does VR improve hard to treat outcomes?
Greenleaf considered that part of the effort is to get all of the brain involved. Need greater immersion, and the user needs to practice reactions to get a lower relapse rate. We need more studies.

Latency to perceive? Psycho-physiological testing shows that this response varies with conditions and takes a long time to acquire.
Luckey offered just try something, the hardware is cheap.
Greenleaf acknowledged that the degree of immersion is important, so staged evaluations with the app is important with layers of immersion.

Killer content for VR?
Luckey mentioned that indie and AAA game developers are looking at the technology. There is interest in the development teams and at the engine developers. VR is wanted.

First mainstream killer app?
Luckey said this is a chicken-egg issue. VR is still something people want to do, but have no clear idea of what it will be. Possibly porn?
Greenleaf suggested gaming 1st, then to other niches. Medical will be big as well as education and training.