Perceptual Tools for Quality-Aware Video Networks
February 4, 2014, Electronic Imaging Conference, San Francisco—Al Bovik from the University of Texas at Austin detailed the efforts in developing automated tools for assessing video quality in video networks. The volume of digital videos is doubling every year, and the computational and network capabilities for the image processing and delivery is struggling to keep up.
In the wireless areas, video now accounts for half of all traffic, and is expected to grow to three quarters of traffic by 2017. This change in wireless traffic now requires a network that is video aware, and can deliver some level of quality video independent of other traffic. Otherwise, the user will revolt.
The cable industry has perceptual quality models and commercial equipment for measuring this quality. The cable and wireline industries have a well defined infrastructure that lends itself to high automation and standards. Most of the OEMs, carriers, ISPs and others in the delivery end of the business are adopting those standards.
The problem is that the delivery standard, SSIM, needs a reference quality as a base model, and all measurement points are relative to that standard. This standard environment is not available for mobile, so a new “blind” video standard needs to be developed that can just use more and better models.
The models need retinal and spatial predictive code, brain and temporal predictions, critical decompression and modulation, and extra cortical models, all of which are behavioral. The vision science models are not like JPG or MPEG, but represent natural visual statistics, which is a low-level data problem. The retina model is a set of spatial linear filters similar to video compression that involves predictive coding, high adjacency, and sparse histograms.
The brain models require predictive coding and temporal processing. The part of the brain that performs this work uses a linear model in a pyramid Gabor transform. The space-time critical decomposition is fairly Gaussian and adds scale and orientation. It is possible to simplify these transforms by assuming a non-linear adaptive gain control and divisive normalization. Summation of the terms with adjacent edges results in a Gaussian distribution and spatial masking.
The motion silencing and temporal masking uses another mask to distort the images with motion. The available tools need better algorithms to improve the results. The motion processing portions of the brain work with a local translation within a neighborhood. For example, block matching motion prediction uses a Huger-Simon cells model. The summation of the magnitude responses that are closer to the visual plane are given greater weight than those further away. The off-plane image segments are more weight in front and less weight behind the visual plane.
These precursors enable instantaneous blind video quality assessments by adding in models of the projected environment and how the brain processes the images. A Gaussian scale mixture model leads to a regular and parametric model. The statistical feature extraction adjusts the parameters to fit into the underlying histogram inviting learning and prediction.
This completely blind model eliminates the machine learning functions. They extract parameters from natural videos with distance for apps to generate quality scenes. The resulting images have good correlation with other compression techniques. The model has to consider time-varying video quality. This behavioral issue has no good models, because memory is one of the inputs to time-varying subjective quality. As a result, scalable video codes or rate switches are more desired. In addition, the reduced reference engine can predict and generate a quality score for the delivered video.
The general flow is to take the various inputs and put them through a non-linear transform. The resulting data are fed to an infinite impulse response filter and back to the non-linear transform to the outputs to model and predict the time-varying subject quality. The non-linear weighting can be trained on existing databases of videos. The results are considered good after a 15 second user conditioning period, and the tools get good results that correlate well with people’s responses. The models are mostly predictable except for periods of buffering.
