GPU real time rendering enables VR storytelling by Peter McGuinness
July 2016: We are now well into the first cycle of storytelling using virtual realityand the creative community is keen to feed back their experiences, good and bad, to the people who build the new tools of their trade. This was a theme at SIGGRAPH 2016.
We have had two great examples of VR movie-making this week: ‘Pearl’, a Google Spotlight production directed by Patrick Osborne which tells the story of a father and daughter road trip and ‘Giant’ from co-creators Milica Zec and Winslow Turner Porter III which deals with the much darker subject of parents struggling to safeguard the emotional wellbeing of a child while trapped in a terrifying situation.
Each of these pieces chose very different approaches: Pearl being animated while Giant is live action, but both production teams zeroed in on a common set of new issues related to moving from single point of view to an environment that the viewer can inhabit inmultiple ways.
While Osborne reminded us that the art of visual storytelling is still about framing the imagery – one reason for his choice of a road trip as his subject was that car windows provided convenient and quite literal windows into his story – he also pointed out that anticipating how the viewer will choose to move around the virtual world is a difficult issue. The story might be focussed on the view through the windshield but if the viewer develops an interest in the contents of the footwell, whether and how to enable that curiosity impacts both the production flow and the arc of the story. Just how far down the rabbit hole can the director allow his audience to travel; and how can he bring them back?
The more naturally contained world of ‘Giant’ might seem to mitigate these concerns but in Zec’s words: “we had to unlearn everything we knew about filmaking” a process she likened to “walking through fire”. In their more constrained visual world, the story became more multi-sensory, with sound being used to augment the immersive experience both by intensifying the main action and also providing cues to guide the viewers’ attention within the 3D space.
The impact of all of this on workflow is profound: a high cost of providing paths into the story via multiple somewhat unpredictable routes will constrain the viability of VR as an art form. When asked for their highest priority, both teams emphasised the importance of real time, fully immersive visualisation tools that integrate as closely as possible with the overall production toolchain and that allow the director the same freedom given to the eventual viewer.
GPU hardware allied to real time rendering closely based on game engines such as Unreal, CryEngine and Unity are set to play a critical enabling role in this process, and we can expect to see a rapid move in that direction, not only as an aid to productivity but also as a foundational technology for VR moviemaking.