Tales from the Trenches: AI Disaster Stories
March 3, 2015, Game Developer Conference, San Francisco—A panel described the failings of AI systems in various programs. Dave Mark from Intrinsic Algorithm acted as moderator, with panelists including Brian Schwab from Magic Leap, Steve Rabin form Digipen Institute of Technology, Marty Singer an independent developer, Kevin Dill from Lockheed-Martin, and Rez Graham from Maxis.
Rabin described an issue that emerged while working on Microsoft’s Baseball 3-D back in ’98. They learned five important lessons; that the team needs to have a b.s. detector to start the project and catch the case of insufficient domain expertise. None of the developers had ever played baseball, so the actions and physics were foreign. Second, the team needs to have good communications, especially between the program manager(s) and game developers. Third, check the tool chain. Microsoft had a very poor custom scripting language with low performance and functionality.
Because of the lack of domain knowledge, batters would only hit to one field and outfielders always threw to the same cutoff person. In addition, the project took over two years to complete, because most of the game was developed in 2-D, so there were no 3-D assets. Half of the AI efforts were expended to make the game look good. And finally, the animations didn’t work because most of them turned into a puddle of polygons.
Schwab worked on a PC game that included a game editor about ’00. When you were trying to edit the game the names of all of the objects were revealed. One of the game developers had identified all objects with vile names, so the players got a vulgar edition in the edit mode. When they tried to cleanup the names, the AI didn’t recognize any of the new names and didn’t work.
On a job for Sony, they were making a basketball game. The problem was that the developers wanted to include every possible play and player action, so the decision tree was enormous. One developer was working on a major portion of the game, and commented out about three quarters of the tree, for easier debug. Three months after he entered the comments, they were getting ready to ship and wanted to know how to get to another playing mode. At that point, the learned about the commented out sections. Fortunately, the AI was robust enough to handle most of the missing sections. The nice thing was that they had over half of the next version already completed. As a result, they suggested that if anyone comments out a section, to clearly identify that section.
Dill was working on some training tools in ’09 on cultural biases. The basic character was an angry granny who was non kinetic. They wanted to create a longer user experience so added more actions and dialog. The granny was supposed to speak Pashto, the language of Afghanistan. The had a role player create actions for mocap and recorded the voice and actions together. Major problem!, the voice actor spoke Farsi. The gesture sequences were used to break up the repetition due to different blends and words.
Singer was working on Shadow Watch, with a 9-month turn for the next generation. Therefore they had a lot for work on the AI. They decided to let the enemy have full knowledge of the level, sot they could circle around and attack from the side. Unfortunately, the players thought the movements were just faked since the enemy would disappear, then reappear in another area. You have to give the players clues about the mapping functions or they don’t believe.
Graham was working on Sims Medieval, which added professions and quests to the simulation game. One NPC had the job of feeding the pit beast. This executioner pushes victims into the pit to feed the beast. One bug let the adventurer make the executioner run into the pit. This is one effect of autonomous functions, they allow something to happen and never repeat. Another problem was the insane trait. It allowed a sim to ignore the actions of any other as well as ignore death. This was fixed to become more reasonable. Some suggestions: don’t bypass existing systems and do use automated testing as much as possible.
