Big memory for new servers at OOW
October 2012 – At the San Francisco Oracle Open World Conference, CEO Larry Ellison pontificated about the new hardware architecture using the latest Sparc64 cores for supporting big memory configurations. The new hardware can support up to 32TB of DRAM in an SMP architecture, and as a result, they have redesigned the database to support the new capabilities for a large single memory load and query.
The systems – called the Exadata X3 – also have a new tier of Flash SSD up to a capacity of 22TB. To feed this IO tier, there is a high speed parallelized PCIe environment that will support up to 100GB/s to the Flash bank. This speed will be obtained once PCIe Gen3 is fully adopted in the design. They includes a special software cache in the 4TB of DRAM per server to support the latency offset in the Flash reads / writes to maintain throughput. These performance figures are currently being addressed with the gen3 Intel Xeon chips which are providing higher performance and lower power for the same costs. Performance will increase as the Sparc 64 cores are added intothe system.
These new large memory systems now bring sub-millisecond latency for OLTP applications, with sub second analysis results. This increases batch processing for standard and custom report generation based on complex query sets, by a factor of 10x. This magic 10x number, makes the large memory rebuild of the software, a “fundamental” rather than incremental change to the industry. As a result of fundamental change, the platform will for the first time allow for same day and real time data processing of complex tasks such “what if scenario analysis” of quarterly and year end financial closing for international corporations. These tasks are typically continuous multi-day (5+) compute events with current method, and under the large memory implementation the data analysis is in minutes to hours. This shift allows for multiple scenarios to be run or verification of the data runs to be completed, when these have never been viable options in the past.
A challenge on these large memory analyses is the system I/O bandwidth for the data retrieval. Typically the data is stored on standard rotating media which still has to offload to the SSD tier. The current SSD tiers are operating a Infiband speeds which, it utilizing a 2X SDR configuration for Infiniband, yield an 8Gbps connection. In order to populate the 32TB of DRAM, this connection for the backplane would require 8.9 hours to load the memory assuming no retries or system latency. With standard TCP/IP traffic levels and multiple retries, a typical load time for this data is 44.5hr or 1.9days. In the overall scope, two days of data load prior to multiple seconds/minutes/hours long analysis runs is still a major breakthrough over the 5+ days per run, however scheduling, backup and restore and UPS planning have to be adjusted as a result of this new configuration.
These servers are being heavily optimized for the OLTP world of very large numbers of small records and are not targeted towards extended run length (i.e. media and video content) files. As a result, metrics collection and account management of media distribution and VOD is very applicable to these new systems, however the serving of the content itself is not.
