ETIA Transmedia Storytelling Panel

June 17, 2014, SMPTE Entertainment Technology in the Internet Age Conference, Stanford, CA—A panel addressed the issues and challenges in developing transmedia for stories. Roland Angst from Stanford opened the session with an overview of the various display modes and their impact on media.

The goal of an immersive feel and interactivity on multiple platforms is getting closer. In the fields of augmented and virtual reality, the changes in hardware are enabling new modes of creation and expression. Entertainment and virtual reality are moving closer together.

Content can be pre-recorded or interactive. Pre-recording is fine for cinema, and interactive is highly suited for training and games. The developments in augmented reality hardware are enabling better integration between the real and virtual worlds. New efforts for 3-D displays in parallax barrier and light-field displays allows the head-mounted displays to become more immersive. The challenges are still in limited resolution, field of view, and low-latency head tracking. The other enabling technologies for this field include improvements in low-power CPUs, sensors, motion, and cameras.

Cameras and projector systems are already being integrated into systems like the Kinect. AR on mobile devices can use location information to extend the capabilities for augmentation. The issues include overlay and camera position, motion tracking, and image recognition.

Interactive storytelling is in games, but must be compromised for the story elements. These elements can be active or passive in an open world. Small story elements are amenable to AR technologies while cinematic elements are better suited for VR. This situation calls for greater support for VR technology development.

The various aspects of vision and storytelling are driving convergence in many businesses, e.g. consoles and services plus original content. The changes are blurring the lines between TV and games. Now it is possible to mix game content and high quality live action shots in either medium. Transmedia takes elements on different devices and fragment those elements for the story. In a coarse fragmentation, the storyteller creates the full story, like in the Star Wars franchises.

A fine-grained fragmentation would take partial elements from the story for an AR game, like a Web-based scavenger hunt that mixes the Web and the real world. The Web in everywhere, so collaboration is possible in location-based games. adding AR functions then makes a lot of sense for this type of activity.

The panel, comprised of Tom Annau from JauntVR, Maya Zukerman from Transmedia SF, and Charles Jablonski a gaming expert, then tackled some issues relating to content creation and game and movie mixtures.

Zukerman opened with comments about transmedia as a new media ecosystem. It broadens the story world for ancillary story lines and changes storytelling while increasing audience involvement. The work depends upon the project, and all projects need to be staffed with multiple disciplines. The creators need to evolve the story and make the story amenable to all media platforms. One key is to test often and use all the feedback you get. You still have to create a good user experience even under changing funding models.

Annau compared and contrasted cinematic VR and gaming. The latest hardware can provide a full 360 degree 3-D experience that can change the user experience from a strong, single point of view to something akin to the real world. The brain enters a new world map versus cinema where the scenes are only in front of you. In addition, you cannot forget the importance of 3-D audio to enable the complete experience. Consumer-level cameras are coming, but for now, only high-cost cameras are available.

Jablonski stated that the technology is still far ahead of the creators. Technology adoption is either a replacement for existing functions or displaces current technology with new ones. Neither of these characteristics is truly disruptive. The resulting platforms still define the user’s perspective. The point of view and the platform are still needed but overall, the story still matters. Creators have to integrate across the technologies.

Zukerman noted that new technology is in the hands of the early adopters. VR is emerging for the second time, and is still not ready for the masses. People will use different media at different times of the day. Changes in technology lead to changes in creativity. Gaming generates greater interactivity, but users want TV to be a lean-back experience. The second screen is not necessary, but can be worked into the content presentation.

Creative tools?
Zukerman responded that technology usually takes a field of dreams approach to adoption. There is no market for some technologies due to the uncanny valley effect (a feeling of disgust or uncertainty when faced with something almost human).
Jablonski suggested rapid prototyping and fast iterations. Creators need to listen to their customers. The creatives need to adopt with an understanding of multiple learning curves. All technology adaptations with creativity have to involve the user and be integrated to make a better user experience.
Annau mentioned that he works with creative people as a technology company. One key is to make transmedia stories on screen anywhere easy to follow. Successful stories are easy to use. The creative vision is much more important than issues of functionality and use.