3-D Cinema to TV Conversions

 January 24, 2012, SPIE Electronic Imaging Conference, Burlingame, CA—two technical teams presented the issues and suggested solutions for cinematic to television conversion for good 3-D. The complementary talks described the basic technology and specific techniques for the conversion processes.

Didier Doyen noted that the problems are due to the changes in size and viewing distance which change the 3-D impressions. Cinema is not the same as TV. The objective, therefore, is to define techniques for the conversion that preserve the cinematic 3-D experience. It’s not just reducing the pixel count from 2k to 1080, but this is the starting place. The images have to have their pixels compressed with width restrictions. On the TV, the maximum depth is greatly reduced from the big screen versions.

The problem, unfortunately, is that the transform is non-linear and depends upon disparity. The disparity transform ideally requires the relative continuity of absolute space. The average image size has a relative connection to the disparity and shrink factor, but it is a very complex and computer intensive process. The alternative is to use a single correlation value for everything.

The conversion process starts with an absolute depth adjustment. By setting the same absolute distance, the resulting images are independent of screen size. Next set the depth within the comfort zone of 3-D TV, from -65 percent to +70 percent of the distance. The depth of field is set for +/- 0.2 of the depth. Then, for the main region of interest, set the disparity for the mean of the sequence or of the depth.

The transform effectiveness was checked by some viewers looking at a 46-inch TV at a 2 meter viewing distance and a 7.5 meter screen at a 15 meter viewing distance. The tests involved both absolute and relative depth and disparity. The overall best results were with the depth averaged with respect to the furthest comfortable viewing position.

For low dispersion video, there was no clear winner in comfort and perceived quality from any of the transforms. The general solution is to perform a single SIFT (Scale Invariant Feature Transform) transform and push the content back onto the screen. The special case of low depth requires no transform. The resulting image are mostly negative from the absolute depth adjustment. The transforms for very small screens in mobile handsets requires better algorithms to be viable.

Christel Chamaret some of the work in raising the levels of automation in the conversion process. Retargeting is necessary as more content moves to smaller screens like tablets and phones. The retargeting conversion to other platforms needs more automatic solutions.

For 2-D video, the conversions are well known and used, but the resulting frame-based images have black stripes due to format mismatch. People universally dislike the black vertical or horizontal bars, and 3-D makes it even worse. The anamorphic distortion or software cropping is the result of the visual attention models or content-aware cropping.

In working with existing image pairs, but not video, they determined the optimal 3-D viewing conditions. The constraints include window violations, color mismatch, vertical disparity, and hyper convergence or divergence. Technicolor has published some guidelines to address the basic 3-D image qualities in their book “Technicolor’s Guide to 3d for Film, TV, and Everything In-Between”

For video, additional constraints include the requirements for no vertical mismatch, popping objects don’t overlap with the retargeted frame border, and the editor has to crop and scale the video inputs to fit into the final disparity range. Retargeting algorithms address the constraints relative to the views in the scene.

The visual attention model creates an object map for not-recommended area of layers for all objects. The maps include depth, objects, and forbidden areas. The flow integrates the various maps and sets crop windows for both left and right images so the combined window locations match. Temporal consistency requires the aspect ratio conversion be added to the depth map and forbidden areas map.

The algorithm checks for proper smoothing, and targets the aspect ratio extensions. The tools need to compute the final cropping windows and overlap to generate a clean 3-D image. These steps take a lot of mathematics and are very compute intensive. This first-pass algorithm needs lots of computers, but does produce acceptable image conversions from cinema to TV.