Where Will Your Next Generation Storage Applications Live?
September 22, 2015, SNIA Storage Developer Conference, Santa Clara, CA—Bev Crair from Intel Storage Group described the drivers and motivations for changing your ideas about storage. The next generation of storage will be a function of innovation, disruption, and trailing-edge technologies.
The primary drivers are: information growth, with data size doubling every two years through 2020; complexity, where we do different things with our data and the data storage structures don’t map to those uses; security, with more attack surfaces and avenues; and the cloud, with new IT solutions and problems.
The latest numbers are staggering. Now, about 306 hours of video are uploaded to YouTube every minute. We are also downloading 50,200 apps and sending 137 M emails every minute. The data use patterns are becoming more complex, moving from collection to analysis. At the same time, more of the valuable data are outside the organization, and much of the data are unstructured.
The data growth is creating more challenges for the IT groups. The data needs are growing much faster than the existing infrastructure and personnel. Storage capacity is growing at a 62 percent CAGR while IT budgets are closer to 2 percent. In ’14, the average IT person managed 230 GB with about 28 M people. Forecasts for storage are for the average to grow to 1,231 GB per IT person with the number of IT people growing to 36 M.
The new data environments of enterprise service providers, server-based storage, high-performance computing, and big data are all moving towards a cloud platform. New protocols are optimized for data tiering, data resident computing, and workload aware platforms. The infrastructure now includes making-based learning, workload data management, and autonomous management.
An emerging alternative to the tiers of memory and storage is among the latest non-volatile devices. Some of these new devices offer some interesting capabilities. Intel’s recently announced 3-D Xpoint devices are claiming speeds of 1000 times that and 1000 times the endurance of NAND, with densities 10 times that of DRAM. The new devices offer some of the best characteristics in performance as measured by delay, persistency, and capacity compared to other technologies.
The next generation non-volatile memories offer a balance of characteristics that make it a viable replacement for the legacy technologies. The NG NVM has the performance of DRAM with higher capacities and non-volatility at a lower cost. The technology is cost competitive with NAND with better retention characteristics.
Looking at the entire range of memory and storage, the SSD NAND provides lower latency and higher IOPS than HDDs. Changing the configuration from a SAS drive to NVMe reduces an additional 20 us of latency and makes most of the system latency a result of software. The NGNVM will offer an average latency of one-tenth of a NAND SSD and about an eighth of the NVMe configuration of NAND.
The advent of new storage technologies affects where your apps live. Silos of storage will work with zones of virtualization in internal networks with scale-up and purpose-built storage. Private and public cloud storage will have nodes working together in external networks which may include compute at the storage site. The storage configurations will be of a scale-out nature or in software-defined storage.
The challenges will be in picking the best technologies for the particular application. The availability of persistent memory will change the architecture of the whole memory/storage subsystem. HDDs will continue to be the workhorse for high volume storage, SSDs will provide performance upgrades in plug-compatible form factors, and NVMe and PCIe will offer even greater performance. The open-source software for software-defined storage will be improved and will be a smaller factor in the overall latency. And finally, the new non-volatile memories will be able to eliminate whole tiers of storage and memory, which will enable even more advances in storage architectures.
