Sensory Gaming Platforms

May 1, 2013, Neurogaming Conference, San Francisco—The new and emerging sensors are enabling new game interfaces. A panel moderated by Jim Gatto from Pillsbury Winthrop Shaw Pittman LLP reviewed the possibilities and considered future applications. The panel members were Stanley Yang from Neurosky, Anders Grunnet-Jepsen from Intel, and Ali Israr from Disney Research.

Yang opened with the migration of games through sensor development. Now developers’ intent is to make machines conform to people for a better user experience. The first apps were for biofeedback for athletes like the US Olympic archery team and the Australian golfers. Both groups need to manage physiological activities through brain calmness.

Newer areas of research include brain training for ADD (Attention Deficit Disorder) and ADHD (Attention Deficit Hyperactivity Disorder). The brain training sequence takes 25 half-hour training sessions. The tracking and progress data are available to the parents and medical staff for evaluation and training modifications.

To make it easier for developers to incorporate brainwaves into an interface, they have a SDK wit 30 EEG parameters.

Grunnet-Jepson in the perceptual compute group is looking at the use of devices to make natural, intuitive and immersive man-machine interactions. Some of the user interfaces are starting to look like “Minority Report”, with gestures, voice, and other sensors for a more lifelike interface. They have a SDK for 3-D vision and another for voice input. Development reference platforms are defined for various interactions.

Israr is performing research into machine outputs like haptics. They are looking to see how perception and physiological responses differ with enhanced output devices. Eventually they will transfer these technologies to the business units for application in products.

One challenge is that of surface area versus location sensitivity. They need to use existing technologies for their research to provide directions for the technologies, but the long-term goals are to incorporate new technologies in products. Other issues are practical versus cost, and personal and effective versus invasive and intrusive.

One new project is a Tesla touch device. The user gets a tactile sensation from electrical fields under the touch surface glass. The sensations can generate feelings of surface texture and hardness. Another project is free-air haptics. This uses nozzles and vortices to generate off-hand sensations. Tools for development are coming.

Perceptions of the markets?
Yang observed that consumer reactions are changing. The first EEG products are getting greater acceptance. People don’t have a problem wearing sensors on the body, but have a reluctance to put something on their heads. The latest product is NeuroCat Ears, low technology, fun. and engaging.
Grunnet-Jepson offered that it is an exciting time. The public is ready for more sensors in gaming and they are willing to try new interfaces.
Israr noted that haptics have been around a long time. The difficulty has been in getting a fine level of feedback to make higher granularity and richer information. By creating new feedback mechanisms, they enable lots of new gadgets.

Is the market ready now? Accelerators?
Grunnet-Jepson noted that the PC form factors are changing with more sizes. The public is willing to try something from the many vendors.
Yang considered that consumers are lazy, and want everything to be easy. Technology advances enable smaller and easier to use devices that can be intuitive. Cost is an issue for some of the platforms, but only if they don’t provide lasting value.
Israr noted that there are few success stories. The proliferation of technologies increases acceptance and experiences. Once the technology exists, people will be willing to try and use it. New interfaces change the user interface and experience.

Next 3-5 years, more changes in the markets?
Israr stated that the markets are growing, but need to have a driver. The new interfaces need context and content to increase consumer uptake. More computing reduces costs. The resources exist and are easily available and in wide use.
Grunnet-Jepson acknowledged exciting hardware. Wireless is making wearable sensors for any platform. The issues are in software and sensor standards to totally redefine the user experience and interfaces. Standards and sensor fusion software with applicable design tools will reduce friction for developers.
Yang agreed that bio-sensors are increasingly being integrated into systems. The wearable devices will take from 1-3 years to get to market, so lots of products will emerge in the next 1-3 years. Gaming is a saturated market, so the new consoles will use more types of input devices.

Why is neuro-gaming important?
Grunnet-Jepson responded that there is a need for more immersive and “smoother” interactions. Neuro sensors can simplify the controller by reducing the number of buttons, increasing the social elements of gaming. Easier is better for interfaces.
Israr stated that natural interfaces increase interactions and immersion. Changing the interface can simplify the game play. New technology allows the game makers to focus on something other than the existing market. Now, both older and younger players can use the natural and easy interfaces increasing the total gaming population.
Yang agreed that wearable interfaces are a tend that leads to more intuitive and easy systems and will attract larger audiences. Games need to be intuitive and fun. Some mental health benefits come form easy and short games with intuitive controls.

Challenges?
Israr offered the lack of content and the technology not up to the requirements. The software doesn’t exist and good APIs and SDKs are rare. All of these tools will benefit game developers. Improvements in technology are relatively easy, but introducing them into the market is hard.
Grunnet-Jepson underscored the lack of apps, distribution, and common platforms all hinder the new developments. Development tools and marketing are needed.
Yang identified the out-of-box experience now needs content and hardware. The challenge is that users want a plug-in and play mode for instant gratification. Their cat ears are cheap, but the video got 1.6M hits on YouTube in 5 days, because people like the price, experience, and good marketing.

Sensitivity at the consumer level?
Israr noted a fine line between invasive and an effective experience. The key element is to design the effects.
Grunnet-Jepson added that the user experience changes with increased functionality and accessories that do more.
Yang stated that with the Mattel Mind Flex, the wow moment came when they provided a new user experience.

Brain interfaces replace dedicated controllers?
Yang pondered the issue and noted that one prototype replaced a RC controller, but needed training and instruction on use. The head sensor freaked out users. Now their SDK offers lots of features and is not technology limited, but user adoption limited. Other interfaces will not replace existing controllers, but will provide an enhanced dimension. New control modes can increase emotion and novelty, but people either totally like or dislike the new interfaces.
Grunnet-Jepson proposed that interfaces other than hands will allow better multi-tasking in a multi-modal game controller. Interfaces will include speech, eye tracking, etc. and are not technology limited. Facial and micro expressions, especially in kids, can be used as game modulation inputs.
Isras noted that increased levels of emotion have a direct correlation with immersion. Providing the sensations of breathing and heart beats can trigger emotions.