Is Spinning Disk Going Extinct? Experts Weigh In
There’s no doubt about it: Spinning disk is getting squeezed. As fast solid state drives (SSDs) get cheaper and tape drives get bigger, the sweet spot where spinning hard disk drives (HDDs) make sense gets smaller every year. But will spinning disks be totally extinct in five years, as some storage experts predict? The jury is still out.
You can find backers of the “Spinning Disk is Dying” camp in many places in the IT landscape, including executives and managers with prominent storage vendors like IBM (NYSE: IBM), Dell EMC, and NetApp (NASDAQ: NTAP).
“I believe the increasing density and steadily dropping cost of SSD technology will quickly outperform mechanical drives from a $/GB perspective in the next couple of years,” Eduardo Rivera, a senior storage architect at NetApp, wrote earlier this year. “When this comes to pass, there will be no incentive to buy HDDs anymore.”
“15k RPM drives are going away,” Gary Albert, business line executive for IBM Storage, tells Datanami. “There’s going to be one more generation [of 15K RPM drives], but we’re not even going to qualify it, because the cost differential between buying that and buying Flash is minimal. Why would anybody buy a spinning disk that has moving parts and is destined to fail, when you can buy something that has no moving parts and is proven to be more reliable, and uses less power?”
“I don’t see much in the way of spinning disk anymore,” says Ed Fabiszewski, an IBM i practice manager with Dell EMC, which recently launched its first entry-level all-Flash VMAX array. “I’m not saying we’re not going to have spinning disk for [VMAX]. But I think what it comes down to, is what you see being configured in the field for customers” is Flash.
The popularity of Flash storage in the consumer electronics space is changing the economics of storage for the enterprise, and that means that spinning disk doesn’t make as much sense as it used to, according to Jeff Barber, the vice president of high storage at IBM.
“The industry is now at the point where the Seagates and the Western Digitals of the world now have two foundries, one for spinning disk and one for electronic memory, AKA Flash,” says Barber, who was trained as a semiconductor engineer.
“Because the scale is getting larger and Flash is a lot cheaper to make them, we don’t have a commensurate growth and expansion of the disk market,” Barber continues “All those economic factors drive the big players in media production to simply their product portfolio, and when they simplify, it’s going to go the way of Flash, because the Flash capacity curve is dropping more rapidly—i.e. how many TB per square inch—than spinning disk.”
Not all storage execs are convinced of spinning disk’s imminent decline. One of those is Tom Lyon, the chief scientist at DriveScale. Lyon’s career has taken him to Ipsilon Networks, where he helped invent IP switching, and to Sun Microsystems, where he helped developed NFS and SPARC.
“As much as everyone likes to bash them, hard drives will remain the only game in town for those with huge on-line capacity requirements,” Lyon says. “Hard drives continue to enjoy at least a 10X cost advantage [over SSDs] for capacity (not performance!).”
Lyon also points out some pertinent data from Seagate (NASDAQ: STX), the $11-billion disk drive maker. He points out that in Seagate’s latest financial presentation, they calculated the five year compound annual growth rate (CAGR) for nearline hard drives at 43%. “Hardly a dying market,” Lyon says. “Nearline means the capacity optimized drives for cloud and enterprise.”
Anyone who has a need for high performance is going Flash, Lyon says. “But if you just need on-line capacity,” he continues, “the 10x cost advantage of the nearline drives is going to make them the storage technology of choice for storing the huge quantities of ‘Big Data.'”
Capacity of HDDs also keeps increasing. In 2016, we saw the introduction of 10TB hard drives. Will we see 16TB next year? It’s a possibility. Regardless, we’ll see denser storage arrays.
The folks at Equus Computer Systems recently developed a 60-drive server with a total storage capacity of more than half a petabyte in a 4U form factor. “This equates to more than 5 petabytes per 42U rack, enabling data centers to store more data in their valuable data center floor space,” Tim Poor, Equus’ vice president of application engineering, recently wrote.
Spinning disks are entrenched in the big data space. Hadoop rose to prominence for the simple way it enabled large numbers of commodity PC servers, with co-mingled compute and storage, to be clustered together into a single entity that behaved like one computer.
But that architectural model appears to be on the way out, as customers look for more agility and speed in their big data systems—as well as the option to use external disk, such as the Equus devices. Companies today are increasingly looking to separate compute from storage—not bundle them together—so they can scale them separately, and take better advantage of advances made in each layer.
One person with a front-row view of the changing storage patterns in big data is Haoyuan Li, the creator of Alluxio, the distributed, in-memory file system formerly known as Tachyon that was developed at Cal Berkeley’s AMPlab. “Disk is becoming the new tape,” Li tells Datanami. “And I’m pretty sure in the future we will support tape.”
Ironically, it appears tape might actually outlive spinning disk as a viable technology (if one were to buy into the narrative that older technologies must die when new technologies arise, which is sometimes but not always true).
While magnetic tape isn’t sexy, it holds a hefty cost advantage over the cheapest nearline disk. As Flash undercuts HDDs as the primary data store, tape is gearing up to make a run at spinning disk to function as the long-term archive.
While the major cloud providers pay to keep hundreds of millions of SATA drive spindles spinning indefinitely, some of them are beginning to adopt tape for the coldest data, says IBM’s Barber. “We can sell them tape at roughly one-eighth the cost” of SATA disks, he says. “The very biggest [cloud providers] have been moving a lot of workload, without customers knowing, to tape.”
There’s even a word for the phenomenon: Flape, where Flash is used for primary storage and tape is used for cold storage. “When we sit back as a corporation and look at what the media market…will look like in five to six years, we’re pretty convinced it’s going be something called Flape,” Barber says. “That’s what we’re seeing in the big cloud providers right now.”
It seems doubtful that HDDs will go away entirely. Of course, it depends on the actual application. While tape may be used to provide access to huge amounts of relatively cold unstructured data, such as video and image files, HDDs will likely have a solid niche serving somewhat warmer data that’s more structured.
Just as tape was pronounced dead so many times, only to stick around year after year, it’s hard to believe that a technology as widely used around the world as spinning disk will go the way of the Dodo bird. For that reason alone, the HDD likely has a few more rounds left in it.