Graphics and Computer Vision
September 10, 2014, Intel Developers Forum, San Francisco—Graphics performance has increased by a factor of 75 since ’06. To keep up with the growing requirements, the latest Core-M processor uses about half of the die for graphics functions. the challenge is to extract full performance from the chips.
To help get better graphics performance, Intel has an SDK for OpenCL that includes development and debug tools based on the SPIR 1.2 release. The toolkit allows analysis of process profiles and also includes Android support. The SDK is an add-on to other development tools, since the main software packages like Adobe and Sony Vegas have unique analysis requirements.
In addition to the OpenCL tools, they also support OpenCV, which can port algorithms from the CPU to the GPU and includes an vision library. This software optimizes the code to match to the hardware and improve graphics performance by up to 3.4 X. Seez, the official maintainer for OpenCL code in OpenCV 3.0, has implemented a transparent API that eliminated the need for specifying CPU or GPU for functions. APIs can operate asynchronously, and the code can improve use of shared physical memory.
These mapping optimizations work for all Intel architectures, Core-M, Iris, and other 4th generation APUs. For the Core-M, the overall architecture is based on execution units (EU), with 8 execution units in a sub-slice. The Core-M uses 3 sub-slices for a total of 8 execution units. OpenCL maps work into SIMD lanes and the processor can be configured as 8, 16, or 32 lanes with a total register space of 4Kb. The register space is 512 bits per work group in an 8-lane configuration. Threads can span execution units and execution unit threads.
Optimizations need to work towards maximum occupancy, to launch enough threads to keep the EU busy while still amortizing the thread launch costs. In short kernels, it is better to use short vector data types and compute multiple processes at a time. Ideally, you want to spread the work around widely enough and keep each thread long enough to balance the launch costs. One example of this works is to change from 1 pixel per work item to 4 pixels per work item.
There are other issues with barriers and some local memory configurations. Sub-slices will not handle partial workgroups for large workgroups. The architecture can only accept 16 workgroups at a time which causes issues for small workgroups. The fixed shared local memory is limited to 64KB per sub-slice, so it may be worthwhile to try other sizes that allow less than 64 bytes per work item.
Memory increases power usage in host to device transfers. One alternative is to try for zero transfers with shared memory. An OpenCL runtime can pre-allocate a pointer that aligns with a page or 4K boundary, so you just have to move the pointer. For image work, a texture sampler is used for sampling pixels. A linear buffer enables zero copy operations by permitting operations directly on the linear buffer with no tiling. The image in the linear store can be used for image resizing and in the pyramid-Lucan-Kande algorithm. In general, it is a good idea to eliminate image copies.
Memory access can be reduces with pipelines. Combining kernels can eliminate intermediate reads and writes between kernels with no intermediate buffers. The challenge is to work with only 2-4 k in the instruction cache. Access patterns also matter. Local memory latencies are similar to L3 cache, which has a 64 –byte line and touches as few lines as possible. In comparison, the local memory may be in 4byte banks for a by-16 structure that has to touch as many lines as possible.
The difference between global and local memories has to be identified and some data need padding to fill a word. An alternative is to use registers as memory to get the highest bandwidth. This area is good for multiple operations on single data. Maximizing compute by avoiding loops and use size and data typing with short data types to trade accuracy for speed.
Many of these optimizations are used in SURF for feature detection and in HOG for pedestrian detection. OpenCL is working on other reductions and scans. The VTune analysis tool helps identify the hot spots in the code to help reduce execution time with a corresponding reduction in power. Some new Open CL features include shared virtual memory, nested parallelism, non-uniform workgroup sized, and generic address space to eliminate the _local, _global, and _constant memory types.
