ETIA Disruptive Innovation in Movies, Sports, and Games

June 17, 2014, SMPTE Entertainment Technology in the Internet Age Conference, Stanford, CA—A panel addressed the changes in movie making, sports, and games due to new underlying technologies. Chris Bregler from NY University and Industrial Light and Magic moderated the panel, with members Marv White from ESPN, Kim Libreri from Lucas Film, and Erik Johnson from Valve Software.

The three fields have different business models. Motion pictures are either live action or animated. The invention of the Rotoscope in 1917 allowed the mixing of the two worlds. Motion capture has moved from hand-controlled to automated and has moved to change gaming. ILM has integrated all these functions into model-based vision using animation and motion capture plus some simulations.

White demonstrated some innovations with video clips. Some of the changes were disruptive while others were only improvements in existing processes. All new ideas need a diverse set of inputs, as new ideas often come from a combination of older ideas, and people come with different backgrounds. Innovation also needs fostering, the ability to make changes. For example, the 1st and 10 marker lines were not well accepted at first, but are now considered a requirement.

The glowing hockey puck allowed tracking, but required a lot of work to survive the rigors of a real game. Stereo is still an experiment that looks to be failing, but still has potential. The ability to mix videos from multiple green-screen sources allowed the interviews to look like they were done in the locker room. The addition of data processing enabled more diverse fantasy games.

Social media is used to track trends and is an on-going development project. The video delivery systems now also deliver games related to the live content. in MLB, the analytics for pitch tracking and other multi-mode tracking allows teams and viewers to see very fine details and analyze all facets of a player. This technology requires a lot of technologies and requires long development times. The technology also has to overcome the slow adoption from the baseball teams.

The “Loom” is a virtual TV wall with many moveable and reconfigurable images. the announcer can pick any image and expand it for different view points, tracking, and other visual aids. The graphics inserts are automatically scaled and repositioned to remain readable even when the image is zoomed.

Libreri talked about visual effects going from cinema to video games. This intersection of games and film uses the growing computer-generated image capabilities and the improvements in hardware make real-time CGI possible. Games in HD use motion capture, subsurface scattering, and virtual lighting for their images. these techniques are the same ones used for movies.

The technologies are also moving the other way, as games techniques are being used in films. Techniques like interactive lighting, visualization, and motion capture have moved from games to cinema. These techniques allow the actors to see where objects are, even if they are in a green screen.

Johnson commented that games are becoming full multi-media events, and film is getting incorporated into the Steam engine to allow user-generated content. The Dota 2 game is a 5-on-5 RPG and now has over 20 M players. A peak of 8 M concurrent players are in the game at any time. The players range in involvement from professionals, to user-generated content producers, to casual players.

Now, they are hosting an international tournament that is broadcast to the world on on-line platforms. The prize pool is over $10M after starting with a $1M first-place prize. The prize pool is augmented with in-game items from the tournament. A Film ” Free to Play” displays the efforts and time needed to host and participate in this game at the pro level. The problems of film distribution are actually much greater than putting on the event.

How to get to success?
Libreri offered that everything has to be a learning experience.
Johnson responded that people matter. You have to hire the best people, and make sure everyone is comfortable with failure as the only way to learn. The gaming industry is a high-risk environment, but learning can make it easier. Their documentary got over 10M views on YouTube, but no one wanted to become the theatrical distributor.
White noted lots of failures. For example, air effects was a measurement of jump height in a basketball game. The problem is that everyone just jumps high enough to get over the rim, and the measurements are not interesting. To be successful, you need to have room for failure, adjust, and restart. You also need the right people.

Why are visual effects, games, and sports drivers for change now?
Libreri claimed that Moore’s law causes all electronics to get less expensive while getting higher performance. Cinema and gaming are merging and in the next decade, sports will be able to do almost anything.

Dota 2 and networks, word of mouth, apps, and the contribution of the Internet?
Johnson answered that the Web is a tool that is necessary for business. It creates a connected user base and allows low friction to users. The Web both forces and enables faster response to issues, and lets the users guide developments.

greater developer efficiency?
Libreri responded that the Web changes views in the real and virtual worlds. The changes will allow other games to move to multimedia platforms. The fans will want to watch the pros in action. There will be more outlets for CGI.
White offered that the end-results are unknown, but changes will happen. We will have synthesized camera men who will be in positions developed from pre-frame time controls.

Video feedback in storytelling move into games?
Libreri agreed that this will happen, and such games are already in production. They share art and other graphics assets, and allow the user to walk into a virtual reality world. This technology allows changes in the story to become a part of the game. It also allows changes in CG and views. The assets such as photo scans and motion capture are used in the build and shared between the media.

Moore’s law, time management, and key technology barriers?
Johnson concluded that the barriers are due to weak imagination.
Libreri suggested bridging the CG and real world. Sensors have higher fidelity and resolution, allowing merged functions and reduced digitizing costs. Adding depth mapping facilitates the creation of hybrid worlds.
White offered light-field technologies to allow greater freedom to create more views from a single image file.

Bregler summarized the computer visioning is an issue, and video effects are not as good as possible.