Data Mirroring Integrity

September 21, 2015, SNIA Storage Developer Conference, Santa Clara, CA—Tejas Wanjari from EMC presented concepts on in-memory data mirroring integrity in distributed systems. The intent is to more clearly identify error sources and error mitigation processes.

New systems are redefining data changes. Disc checksums are for older data and now undetected corruption is not acceptable in many functions like backup and recovery. The corruption sources are still in the same places: system failures, hardware failures, disc failures, process corruption, and network corruption. The problem is that many of the various components in the system were developed when a BER of 10-12 was sufficient.

Now, a single drive can hold over a terabyte and storage systems have capacities of petabytes. the older checksum protocols used weak 16-bit checksums that are now highly vulnerable with the data volumes in use. The statistics say that there will be at least 1 TCP error in 20 GB. At the system level, the checksums are better, but the challenge is that the data/file/ and network structures don’t match. This situation requires some translations which add a performance hit and still leave the network issues.

The ideal solution is to have a zero-copy structure where the data is never moved. All copies have useful checksums to complement the hardware redundancy. Approaching the ideal is RFC 2835 which defines a TCP MD5 signature option. The protocol will retransmit on any checksum mismatch, snce the receiver will not send an ACK. The client and server need to have the IP, port, MD5 key for the other end before a TCP connector is established. The client will have to bind() to the server to save <IP, port, MD5key> mapping.

This higher data integrity protocol causes a small latency hit, but improves throughput for larger packet sizes in a single thread environment. This same improvement tracks multiple threads up to 50 threads and overall memory footprint due to fewer inactive threads. The MD5 protocol is good for some functions like line-speed mirroring of in-memory data, but is not a good generic function. The increased throughput and smaller memory footprint may compensate for the higher latency.