Emotion Prints: Emotion Visualization on Multi-Touch Interfaces
February 9, 2015, Electronic Imaging Conference, San Francisco—Daniel Cernea from Technische University in Kaiserslautern discussed interaction-driven emotion visualization on multi-touch interfaces. The intent of the work is to create better user interfaces and incorporate other human factors to enhance collaboration.
The detect user emotional states allows researchers to measure user affective states. In group settings, three factors are important: self awareness, awareness of others, and group awareness. Self awareness is essential for creativity, motivation, problem solving, and decision making. Awareness of others is needed for collaboration, cooperation, communications, and other social dynamics. And the group cannot function if it is unaware if itself.
The researchers are detecting emotional states with commercial EEG devices as these headsets are reasonably inexpensive and allow for wireless capture of data. Although the sensors can detect and measure many parameters and emotions, they only used valence and arousal since these were deemed most critical and adding more emotive states creates major complications in analysis.
The emotive states were mapped to touch locations and accuracy against the emotional state. This mapping gave a hint of the emotional state as a functions of time and space in two applications, a soccer game and a collaborative work group. The mapping changed color and shape with levels of emotion, and displayed this information on the touch screen.
When a subject’s attention faded, a second user could detect the attention shift and immediately replace the disinterested person. The post-task analysis showed the temporal and spatial distributions of various users and allowed the developers to measure the result of changes in the user interface on attention and performance. The use of emotion prints improved user awareness of their own and other peoples’ emotional state. The affective information helped those playing soccer by giving an advantage to the player who wasn’t discouraged by the game mechanics, and the collaboration tasks saw better problem solving.
The research also raised concerns about correlation and the need for better tracking. The challenge is that the work surface is a large horizontal flat panel display, so eye tracking of moving people is very difficult. In addition, they found a need for greater precision in getting the user’s emotional state, even with only two parameters. Privacy is a large concern as this can be much more invasive than social media, so users are allowed at the start of a session to define the amount and type f sharing in the user interactions. And finally, the issue of security needs to be addressed when accessing a person’s emotional state in any setting.
