Subtitled: Data-at-rest encryption compare and contrast, Netapp FAS & VNX2
So every once in a while I run across a situation at a client where I get to see the real differences in approach between technology manufacturers.
The specific focus is on data-at-rest encryption. Encrypting data that resides on hard drives is a good practice, provided it can be done cost-effectively with a minimal effect on performance, availability, and administrative complexity. Probably the best reason I can come up with for implementing data-at-rest encryption is the drive failure case- you’re expected to send back that ‘failed’ hard drive to the manufacturer, where they will mount that drive to see just how ‘failed’ it really is. Point is, you’ve still got data on there. If you’re a service provider, you’ve got your client’s data on there. Not good. Unless you’ve got an industrial-grade degausser, you don’t have many options here. Some manufacturers have a special support upgrade that lets you throw away failed drives instead of return them, but that’s a significant bump up in cost.
OK, so now you’ve decided, sure, I want to do data-at-rest encryption. Great! Turn it on!
Not so fast.
The most important object in the world of encryption is the key. Without the key, all that business-saving data you have on that expensive enterprise-storage solution is useless. Therefore, you need to implement a key-management solution to make sure that keys are rotated, backed up, remembered, and most importantly available for a secure restore. The key management solution, like every other important piece of IT equipment, needs a companion in DR in case the first one takes a dive.
Wait, secure restore? Well, what’s the point of encrypting data if you make it super-simple to steal the data and then decrypt it? Most enterprise-grade key management solutions implement quorums for key management operations, complete with smart cards, etc. This helps prevent the all-too-often occurrence of “inside man” data theft.
NetApp’s answer to data-at-rest encryption is the self-encrypting drive. The drives themselves perform all the encryption work, and pass data to the controller as any drive would. The biggest caveat here is that all drives in a Netapp HA pair must be of the NSE drive type, and you can’t use Flash Pool or MetroCluster.
Netapp partners with a couple of key management vendors, but OEM the SafeNet KeySecure solution for key management. Having the keys stored off-box ensures that if some bad guy wheels away with your entire storage device, they’ve got nuthin’. It won’t boot without being able to reach the key management system. SafeNet’s KeySecure adheres to many (if not all) industry standards, and can simultaneously manage keys for other storage and PKI-enabled infrastructure. I consider this approach to be strategic as it thinks holistically- one place to manage keys across a wide array of resources. Scalable administration, high-availability, maximum security. Peachy.
I had the opportunity to contrast approach this against EMC’s VNX2 data-at-rest solution. EMC took its typical tactical approach, as I will outline next.
Instead of using encrypting drives, they chose the path of the encrypting SAS adapter – they use PMC-Sierra’s Tachyon adapter for this with the inline ASIC-based encryption. So it’s important to note that this encryption technology has nothing to actually do with VNX2. More on this architectural choice later.
Where the encryption is done in a solution is equally as important as the key management portion of the solution- without the keys, you’re toast. EMC, owner of RSA, took a version of RSA key management software and implemented it in the storage processors of the VNX2. This is something they tout as a great feature- “embedded key management”. The problem is, they have totally missed the point. Having a key manager on the box that contains the data you’re encrypting is a little like leaving your keys in your car. If someone takes the entire array, they have your data. Doesn’t this go against the very notion of encrypting data? Sure, you’re still protected from someone swiping a drive. But entire systems get returned off lease all the time.
Now, of course if you’ve got a VNX2, you’ve got a second one in DR. That box has its OWN embedded key manager. Great. Now I’ve got TWO key managers to backup every time I make a drive config change to the system (and if I change one, I’m probably changing the other).
What? You say you don’t like this and you want to use a third-party key manager? Nope. VNX2 will NOT support any third-party, industry-standard compliant key manager. You’re stuck with the embedded one. This embedded key manager sounds like more of an albatross than a feature. Quite frankly, I’m very surprised that EMC would limit clients in this way, as the PMC-Sierra encrypting technology that’s in VNX2 DOES support third-party key managers! Gotta keep RSA competitors away though, that’s more important than doing right by clients, right?
OK. On to the choice of the SAS encrypting adapter vs. the encrypting hard drives.
Encrypting at the SAS layer has the great advantage of being able to encrypt to any drive available on the market. That’s a valid architectural advantage from a cost and product choice perspective. That’s where the advantages stop, however.
It should seem obvious that having many devices working in parallel on a split-up data set is much more efficient than having 100% of the data load worked on at one point (I’ll call it the bottleneck!) in the data chain. Based on the performance hit data supplied by the vendors, I’m probably correct. EMC states a <5% performance hit using encryption- but has a caveat that “large block operations > 256KB” and high-throughput situations could result in higher performance degradation. Netapp has no such performance restriction (its back end ops are always smaller), and the encryption jobs are being done by many, many spindles at a time, not a single ASIC (even if there are multiples, same point applies). However, I could see how implementing an encrypting SAS adapter would be much easier to get to-market quickly, and the allure of encrypt-enabling all existing drives is strong. Architecturally it’s way too risky to purposely introduce a single bottleneck that affects an entire system.
Coming to the end of this rant. It just never ceases to amaze me that when you dig into the facts and the architecture, you’ll find that some manufacturers always think strategically, and others can’t stop thinking tactically.