How Do We Handle All the Data
November 27, 2012, Server Design Summit, Santa Clara, CA—Andy Walls from IBM
talked about server-storage problems and how the many different forms of data are making the problems worse.
The information explosion is overtaking the whole industry. We are generating over 2.5 exabytes a day and 90 percent of all data has been created in the last two years. The data generation trends are expected to continue and even increase, as distributed computing and the internet of things takes data generation to a new level.
For example, a video surveillance system for traffic might have 400 intersections instrumented. These intersections will generate 790 M files amounting to over 300 TB. The system generates 2.6 TB per day or 300 files per second. The workload for this system is 95 percent write IOPS.
Medical imaging generates similar volumes of data even as the creation and analysis of those data have contributed to a major change in life sciences and health care. When text, data, analytics, and accelerated processors are used to evaluate brain aneurysms, the diagnosis gets a 85 percent correctly detected rate compared to 45 percent for a manual analysis.
Capacity and velocity are the keys to these data. Users need to view the analytics at near real time to derive value. All industries need to become IT industries to get a competitive edge in today’s markets.
Flash lcan reduce the I/O bottleneck as the interfaces like PCIe, and RDMA get better. Faster switches in the networks enable new access models. The hardware is changing to increase the throughput, but the I/Os have been optimized for HDDs and not flash. There is a great need to reduce the processor overhead for a store operation. Other I/O operations are also increasing as the data moves to many locations in the system.
The change to non-volatile memory is not just a technology issue, but also involves economics and business issues. The IT industry needs consumer applications to drive down the prices of flash storage. This consumer pull has already reduced the costs of some media formats like USB drives and SD cards by generating enormous increases in flash memory demand. This pull will also apply to other formats over time.
But there are many known and unknown issues in transferring flash to the data center. The latency times will improve by a factor of 10 as will the IOPS. But the move to flash will impact server and network architectures, since path lengths are not optimized for SSDs. Data and databases are not laid out for SSDs, so the data has been moved to the DRAM cache to prevent HDD access for performance issues. Another problem with most flash storage is that the drivers are not scalable to multiple cores. These bottlenecks can change, but it will take time.
HDDs can provide high throughput if you are starting on the right track or are in a streaming mode, but the IOPS are still fairly low. Other performance optimizations for HDDs include grouping I/Os together to avoid I/O. For systems with SSDs, the small block and random I/O performance is good. Due to the available performance, the system could stop avoiding reads and over read if possible. I/O code paths and change access parameters to increase symmetric multiprocessing scale.
Products will drive other changes to the interfaces, and lower latency memory will invoke other changes in architectures. Currently, about 80-85 percent of storage, minus the boot drives, is shared with the balance direct attached. In the near term, we will see more direct attached storage used for cache and other acceleration functions. this level of storage will be connected to the servers for the hot data.
Nevertheless, sharing is still important in big systems. An alternative is to cluster storage at some levels to take advantage of locality. Although RAID controllers are optimized for HDDs, they can be updated for SSDs. Even with solid state storage, there is a need to have redundancy for data protection.
