ISSCC 2013 Session 9 Mobile Application Processors & Media Accelerators

February 18, 2013, ISSCC, San Francisco—Session 9 reviewed the area of small processors. Contributors included Samsung, Renesas, National Chung Cheng University and Industrial Technology Research Institute and TSMC in Taiwan, MIT, California Institute of Technology, University of Michigan, KAIST from Korea. These processors are intended for phone and tablet applications, so must meet the conflicting requirements of low power and high performance.

Yonghu Park from Samsung presented a fused multimedia processor with an ARM CPU and a general purpose GPU designed for multi-format video codec processing. The 72.5 GFLOPS chip integrated the various functions for increased efficiency over separate instantiations. The codec handles 240 M pixels per second in a full 1080 P 60 fps stream.

The chip has 5 CPU cores and an imaging subsystem that accelerates the video processing. The multi-core processor can handle inter-core pipelining and on-the-fly memory to memory transfers. This improved memory interface is deigned for energy efficiency and the whole design uses dynamic voltage-frequency scaling to further minimize power consumption.

The chip uses task, pipeline, core, and scalability parallelism and takes advantage of the 12.8 GB/s memory bandwidth. The complex power management used separate power domains for each CPU, GPU, memory, and the other logic. In addition to the voltage and frequency scaling, the ship uses extensive power and clock gating for maximum power reduction.

Rahul Rithe from MIT described a reconfigurable processor for computational photography. The processor is targeting the high dynamic range (HDR) image processing functions. This process makes a composite picture from multiple images with different exposures to display a much higher dynamic range than is possible forma normal, single image. The outputs can be a tone-mapped HDR image, a low-light enhanced picture that combines a flash and non-flash image, and any other image needing extensive filtering.

The basic building block for the reconfigurable processor is a bilateral filter, comprised of a grid assignment block, a convolution engine, an interpolation function, and all interconnected through a 3-D data structure called the bilateral grid. The image processing fractures the image into intensity grid points and bins the sub-grids for intensity and weight.

The grid filtering functions uses 27 parallel processing blocks with 3-stage adders. The outputs feed the 16 convolution engines and the buffered outputs go to the linear interpolation processors. All of this filtering takes a lot of processing and storage, requiring memory management to keep the functions operating, and move data in and out of active memory and the function blocks. The design uses 65 MB of memory to process a 10 M pixel image.

As implemented, the chip reconfigures the processor depending on the tasks at hand. HDR image processing takes three or more standard definition images at 8 bits and transforms them into a single 16-bit HDR image. The functions and algorithms for HDR creation have to output tone mapping to transform the 16 bits/color to a lower number of bits for display or other output. Similarly, low light enhancement merges a flash and non-flash image. One area needing more work is in shadow reduction, as the flash image has more and darker shadows than the low light image.

The processor implementation in 40nm CMOS operates on supply voltages form 0.5 to 0.9 V with 1.8 to 2.5 V I/Os. At 0.5 v, the processor runs at 25MHz, increasing to 98MHz at 0.9 V. Run times are comparable to a Nvidia NV40 GPU at a fraction of the power, and over 15 times faster than an Intel i5 dual core processor running at 2.5 GHz.

A feature extraction accelerator is focused on autonomous vehicle navigation. Computer vision is becoming more widespread, but power constraints prevent greater proliferation. One important aspect of computer vision is feature extraction to find useful information from a video or image. Feature extraction is a necessary precursor to object recognition, pose estimation, and other functions.

The challenges in processing real-time video is a need for very low power, high compute requirements, high variability in throughput, and the need for large buffers. A proposed design has a 2.7 mW power budget and a generalized feature extraction engine. Conventional approaches use multi-core architectures and an application-specific front end that is highly dependent upon preprocessing.

This proposal uses highly parallel hardware accelerators and low power FIFOs and an algorithm-architecture co-optimization. This new system uses an feature extraction engine on an image for comparison to a database of known objects. This set of features is used to construct a map and the vehicle then uses a set of other sensors to navigate the map.

The video flow is a 30 fps 8-bit VGA greyscale image set that is subdivided into 11 sections of 640 X 124 overlapping horizontal slices. The full architecture includes a 2-D image integrator, Gaussian box filters, local maxima detectors, wavelet filters, descriptor processing elements, and a post processor to generate a single stream of designators.

One circuit change was to develop a low-power, near-threshold FIFO based on a latch-based shift register. In this design, data move one stage per cycle, and there is one output per n cycles. In addition, one latch always has a duplicate value that bubbles back to the originating latch. A balanced leakage MUX allows a single output per cycle throughput, and data lanes are activated on alternate cycles.

The SURF algorithm was modified for low power operation and the optimizations also reduced readout delay and improved energy efficiency. The algorithm changed the original square sampling region to a circular sampling region with 11.25 degree rotational steps. The modified SURF provides almost identical results to the original algorithm. All together, the optimizations resulted in the following savings: parallelized filters- 38 percent, single stream and circular-shaped descriptor-48 percent, shift-latch FIFO-22 percent, and optimized detector-27 percent.