When Art and Games Collide
July 22, 2014, Casual Connect Conference, San Francisco—Kelly Fennig from the National Film Board of Canada described his efforts to use gaming tools and conventions to make an art exhibition more interactive. The challenge is that the transformation of a photo exhibit into a “non-game” requires also making it accessible for non-gamers.
The idea of a non-game was to get greater interaction with the art and stories associated with the exhibit, a look at images from 1948 Vancouver. They created an iOS app to deliver non-linear audio that was associated with the photos to become an immersive environment. Their tools included Maya and an open-source tool called Kraken.
The biggest differences between games and interactive art are that the art has no set rules for starting and stopping points, but is all freewill driven. There are no penalties for gong in a different direction and the risk-reward is the best use of a person’s time. In addition, there can be no functions that create frustration and no right or wrong way to experience the exhibit. The only outcomes possible are like or don’t like the show.
As a result of these conditions, they had to design different user interfaces to ensure a pleasant user experience. The drivers for any game are in the personality, motivations of the player as well as prior exposure to games, control familiarity, and purpose of any functions. the interactions have to start quickly to get the user’s interest, then slow down so they can absorb the feelings in the pictures.
The controls have to be intuitive, so a WASD interface or game controller are not appropriate for the 80 percent of non-gamer attendees. The developers wanted to use minimal text and have very simple menus. The resulting gamified design uses audio and visual aids for general functions and had to anticipate the unexpected as well as be relatively dummy proof.
When the app starts, it takes you into the art world in less than 45 seconds. Most controls use 1 or 2 buttons, and the buttons are very large, half of the screen. The button functions are integrated into the tablet display and invisible to the user. They investigated a number of control schemes for the tablet and found that a tap to move is bad for most users. A virtual joystick seemed too “gamey” and a viewport into the scene seemed too much a gimmick and also tended to drive viewers to look towards the floor rather than at the display presentation.
They settled on gestures using one or two fingers for all functions. The embedded game loops allowed visitors to explore the photos in any order and allowed them to listen to the various stories associated with an location or not. All instructions are limited to one screen, and most of the information is included in various iconography. The instruction screens are dismissed with a single tap.
The user interface had minimal menus to keep from confusing new users. A tap on the menu button resulted in a 4-button ribbon: location and story selection, input control toggle, credits, and additional information via a link to a separate website. Navigation included sight and sound cues, with key areas and directions highlighted with brighter image sections and function-specific tones. At the same time, visitors are allowed to go to any part of the exhibit, there are no keep out areas. However, you cannot drive yourself into a corner and get stuck.
This cause-and-effect interface allows for greater interactions, allowing visitors to follow one story at a time in 15 locations. Audio and visual stimuli like glowing shadows, sprite clouds, characters in the scene, and alert tones draw people into the environment and notify them to trigger the associated audio tracks. The sights and underlying storytelling allow free will and pacing and only suggest leads to a single track and a path through the exhibit.
Due to the free flowing nature of the viewers, collision detection and tablet identification had to be very robust to keep the various sound tracks aligned with the individual viewer. There were many tradeoffs to reduce frustration, since non-gamers are easily frustrated, they just won’t keep grinding to get to another level. The system has to be fast loading and the controls have to be responsive and consistent.
This exhibit is just an adventure, with no bosses or battles it is geared for the 80 percent of people who are not game players, yet have access to tablet computers. The installation required the cooperation and collaboration of artists, writers, and game developers to make a museum display work like a game.
