Sound for the Small Screen
October 11, 2014, Audio Engineering Society Convention, Los Angeles—A panel considered the issues associated with smaller screens like tables and smart phones. The targets are games and small videos and music on the mobile devices, which require changes in the work. Jerome Rossen from Freshmade Music moderated the panel, with panelists Steve Horowitz from the Code International, Richard Warp from the Manhattan Producers Alliance, and Rick Torres from 17.
Horowitz commented that the mobile and social areas have the issue that you don’t know where the content will be consumed. For a project for Nickelodeon called “Doodle Jump” they needed the sound of Sponge Bob as well as other sounds for navigation and game play. A real guitar constitutes most of the background music. Some issues relate to the branding and to the nature of the game.
Kids’ games need to express happiness and the target audience can be finicky and easily lost. They also need to provide only positive feedback to keep the kids engaged. Mobile games have a lot of content with big files for music, voice over, and sound effects that are all triggered by the game engine. The challenge is setting a good mix for an unknown environment.
It is very important to test the various implementations on all platforms. The many variations of the platforms have significant performance differences. An iPad has speakers that are close together and a small dynamic range, very different from an iPhone with even more limited sound capabilities. A pre-mix in surround sound can be used to test most levels and allow for listening in a known environment. The end device can handle the conversion from surround to stereo or mono. One convention is to normalize the content at 90 percent and balance all else to that level.
Codec choice is important. And MP3 is not good for music because the format inserts a gap at the start of any file, but it is ok for voice. An alternative is the ogg orbis codec. AAC allows a seamless loop. You have to match the file types to the intended platform. The loop length depends on the place in the game, so you have to map the loops to the game play mechanics. The issue is file size versus loop repetition rate.
Warp talked about experiences wit Leapfrog. They use a lot of small footprint audio that is designed for poor speakers. The outputs take a lot of iteration and rebalancing for the mostly mono music. The educational focus and constant reinforcement sets the production values.
In another project, the limits due to capture on a iPhone mike were a challenge. The display and playout uses layers of sound, music, effects, and voice. They have to fit the audio to the images and not overload the senses. The sound and music were added to “selfie” images to create a movie. Another project filmed Nepalese refugees. They sampled the indigenous sounds then mixed them down to become background music and ambient sounds. The big issue was how long to loop a sound segment.
Torres described the challenges of recording and mapping audio to video when using other languages. This project was to promote government-individual interactions and was dubbed in a number of languages. One trick was to find and use the native cadences and the script supervisor to determine transitions and mix cues. The challenges in voice overs or overdubs in multiple languages is that English is a fairly compact language and the pace changes. To get the words to fit the images and getting something close to lip sync is very hard.
One possibility is to record everything and place the audio as close to the sync position as possible. Try to work quickly and understand your vocal chain so you can get a close to final version on one pass. The difficulty is that this type of work becomes part of on organizational function. It requires flexibility and the ability to work with a script supervisor. The difficulties are that the different parties in the creative functions have conflicting priorities. The script supervisor wants the message to be in proper grammar in the language while the end customer wants the finished product to be under legal approval, so changes in words and expressions may not pass muster.
