I thought I knew quite a bit about enterprise storage.
I studied computer science in school; we learned a lot about L1 and L2 cache, cache hit/miss ratios, and the amazing differences in speed between L1, L2, DRAM, and disk. And years later, we saw the revolutions in storage area networks, virtualized storage, and flash. Only a few years ago, I ran fairly large data centers with over a petabyte in enterprise storage units using SAN Volume Controllers.
But you know how it is in the computer industry; you get distracted for just a moment and the whole world changes. It’s as if you’ve got these kids named Cloud, Network, Security, and Storage, and you turn your head for a minute to yell at the other three and when you look back, Storage has paint all over the wall. That’s how I felt listening to Dr. Axel Koester and Clod Barrera at the Storage Systems Technical Kickoff at IBM Edge 2015, except that Storage has painted a really nice picture in this case.
It seems as though SSDs just got here, but Axel and Clod told me that they’re now obsolete. Note that this doesn’t mean that flash technology is obsolete – but packaging up the flash chips in a little box that looks like a hard drive and having the computer pretend that there’s still a spinning metal platter inside there is not the way to go. With SSDs, you only get 50 gazillion times more speed than hard drives, and when using flash directly, you get 200 gazillion. I may be slightly off on the numbers.
To me, the lines between cache, ram, and disk are all blurring. We still have volatile memory (the stuff you wish you’d saved when the power goes out) and non-volatile (the stuff you wish you’d backed up), but beyond those distinctions it’s getting hard to tell. (My security personality feels compelled to point out that even “volatile” memory might last longer than you think with the power off — google “cold boot attacks” some time).
Some systems have four levels of cache, two different types of RAM (fast and slow), and now varying speeds of “disk” (which may be flash or spinning platters). And beyond that, on the network you have fast object storage and slow object storage (“nearline”), and really really slow storage which may take minutes to retrieve a piece of data but is extremely cheap. There are charts, papers, and algorithms to help you pick the right storage for your particular workload.
On the slightly different topic of data density, this slide made me giggle.
Apparently, while you can read extremely narrow tracks of data, the write heads plough a huge, wide swath through the magnetic media — like me trying to use the riding mower where I should be using a weed trimmer. One solution researchers have worked on is to write one layer, then the next mostly overlapping that (so that only a small part remains readable), then the next mostly overlapping that, and so on like roofing shingles. The next time I look up, Storage will be off to college.
Sarcasm as a Service 