Unpacking the Storage Dilemma

This blog is an excerpt from GovLoop’s recent industry perspective, The Storage Dilemma: Flash, Hybrid or Disk. Download the full perspective here

For federal agencies, today’s data explosion comes with a huge and complicated mandate: Gather data, store it efficiently and, above all, protect it. But how?

Between budgets in flux, lagging legacy IT systems and the widely discussed and ongoing IT brain drain, the odds often seem stacked against federal IT teams. But data waits for no one—governments included—nor do today’s rapidly changing technology or the cyber criminals looking to infiltrate it. Can your agency possibly keep up?

It has to, with cybersecurity and data-loss prevention playing an ever more prominent role in our national dialogue. More than ever, government agencies need smart data storage solutions that blend efficiency, value and performance. The first step? Deciding which solution best meets your agency’s needs: all-flash, disk-based or hybrid-flash storage.

Disk vs. Flash

Flash storage, simply put, is any type of data repository or storage system that uses flash memory—a compact, flexible storage medium that can retain data in the absence of a power supply and that can be electronically erased and reprogrammed.

Disk storage is well-suited for housing large amounts of data, particularly when that data is at rest. While it may be the cheapest option up front, it does consume substantial space and electricity within data centers. Traditional data storage uses disk arrays, which can drain space, power and money, in addition to being unable to easily handle big data.

The breakdown: multi-tiered flash storage can address those issues. Instead of writing data on disks, flash storage uses flash memory to store the data, thus reducing space requirements while increasing performance capabilities.

Flash also provides improved storage performance, with high input/output operations per second (IOPS), consistent submillisecond latency and better bandwidth. These features are essential for databases, analytics, backup and cloud services applications. Meanwhile, flash offers a variety of other benefits for the public sector and IT managers, including:

• Excellent primary storage performance with consistently low latency
• Less power consumption
• Lower failure rate
• Deduplication and compression of that data at the array level, which assists in mitigating unmanageable data growth.

“The deduplication and compression features allow for a much smaller data center footprint, resulting in OPEX savings on rack space, power and cooling,” James Hebert, a solutions architect at Force 3 said.

For example, he explained, a storage system with a configuration of 1 Petabyte (PB) effective usable capacity would typically require nearly two full cabinets (70U) if using 10k rpm spinning disks.

Conversely, with an all-flash array, that footprint decreases to 5U of space, resulting in 14 times less rack space and weight. In addition, the all-flash array uses 13 times less power and cooling than a traditional array with spinning disks only.

Hybrid Storage Arrays vs. All-Flash

Unready to immediately transition away from disk-based storage, some agencies are opting for a hybrid approach.

Hybrid storage systems blend flash-based solid state disk and hard disk drives to provide high performance at an affordable price. In doing so, they attempt to address the dollar-per-IOPS vs. dollar-per-GB decision that IT professionals often grapple with.

“A hybrid storage platform consists of both flash and spinning disk storage,” Hebert said. “This allows for the best of both worlds: flash storage for workloads that need it for data caching, and spinning disks for capacity workloads. Many organizations use this approach because it’s more cost-effective, since not all of their workloads require flash performance.”

An agency might typically choose hybrid flash arrays over all-flash for any of the following reasons:

• They have multiple workloads, some requiring flash and others not
• They run applications that aren’t sensitive to latency
• They don’t need extremely high IOPS

“An agency that opts for an all-flash array would typically need high input/output operations or consistently low latency,” Hebert said, “such as that needed when deploying VDI or online transaction processing databases.”

Another good reason to choose all-flash over hybrid flash arrays, Hebert said, is that some workloads greatly benefit from the deduplication and compression capabilities, such as virtual server (VSI) and desktop (VDI) infrastructure deployments.

“Given this benefit, much less storage will need to be purchased, resulting in space, power and cooling savings,” he said. “In addition, the performance and latency will be consistently better, resulting in an overall better user experience.”

Still, there are pitfalls to choosing a hybrid flash approach over an all-flash storage array, Hebert said.

Hybrid storage arrays use flash devices to cache “hot” data—a term typically used for blocks of data most often manipulated by applications using the storage system, Hebert explains. These blocks get “promoted” onto flash drives, making access to them much faster because of their consistently low latency and speed, compared to spinning disk.

This causes an all-flash type of performance for that data set. But complications appear when the applications finish manipulating that data set and begin working on another with completely different data blocks that currently reside on slower spinning disks.

The problem? Promoting these new blocks into the cache and demoting the finished blocks onto spinning disks takes time. Consequently, the transition causes inconsistent latency patterns. And, because some applications are very sensitive to latency, this may cause the application to slow down or even crash, depending on the size of the data set and other factors.

To learn more about how to solve your storage dilemma, download GovLoop’s recent industry perspective, The Storage Dilemma: Flash, Hybrid or Disk.