PCI-Express for Storage Applications

August 18. 2015, Intel Developer Forum, San Francisco—Dan Froelich from Intel described the storage transition to PCI-e interfaces. The change is facilitated by new form factors and products.

The availability of storage in SSD and PCI-e configurations is approaching 10 percent of the total storage market and growing faster than rotating disks. Forecasts indicate a crossover to half of all storage will be semiconductor-based and on a PCI-e format, not magnetic media in late ’17 or early ’18.

Part of the driving force is the many form factors available for the storage. The PCI-e specifications identify an electrical interface that gives requirements for the PHY, compliance and interoperability, and various packaging. The spec allows the drives to be configured for a range of power, performance, and data rates. In addition, the interface can be setup for low power operation.

The current form factors include BGA packaging, a small board called M.2, a 2.5-inch hard drive enclosure called U.2 ( formerly SFF 8639). and a plug in long board. The U.2 form factor allows for direct connect and hard drive replacement and also allows hot swap. Systems can be set up to plug the enclosure into a drive slot or can be connected via a SAS cable. Some topologies can be hard to implement due to electrical issues and limits to cabling lengths. Existing mini-SAS cables are ok for the existing PCI-e specs, but will not be able to meet the requirements for the next spec revision, which calls for clock synchronization.

The cable issue is being addressed with a new cable standard called OCuLink. The reference clock for rev 4 will need to accommodate a spread spectrum clock for lower EMI, so the clock and data recovery functions have to be present at the receiver node. This mode of operation is different from the current common clock. The advantages of the SRIS spread spectrum clock include fewer clocks, no special clock routing, less shielding, and no length limits. The disadvantages are slightly lower performance of about 2 percent, and reduced receiver margins due to jitter.

These issues only apply to designs that use an external cable. The system will need some automatic detection fro SRIS and common clock modes. For on-board storage, the BGA package is the smallest footprint and volume but will more difficult to service, even with sockets. The PCISIG is looking into an even smaller BGA package for the rev 4 spec. These configurations are directed towards the mobile devices which have little available space. The small card M.2 uses a standard PCI connectors on the motherboard or backplane and can be used in a larger device like a tablet, or larger platforms.

The conformance and interoperability tests are currently only configured for the large plug-in boards, so the SIG is working on updating the specs for the U.2and M.2 form factors to identify test fixtures and setup procedures for the new packaging. The existing documents for test procedures will continue to be in effect for electrical and performance testing.

Test calibration will define the link topologies and will call for retimers to extend the length of the links. The receiver tests are hard to run, so the testing will change to more of a receiver margining test rather than the L0 operating state. The latest test proposal will call for newer drives to have margin control hardware and software to allow incrementing or decrementing the “0” and “1” state voltages to a usable minimum threshold for the receiver testing. The 4.0 rev is expected to be released later next year for product to appear between ’18 to ’20.