Earlier this week, Western Digital announced the Zoned Storage Initiative, an initiative focused on data infrastructure, and more specifically on open data infrastructure designed to take advantage of Zoned Storage Devices (ZSDs).

As more and more data will be sequential in nature – video, IoT/edge data, surveillance, or data that can be grouped or "zoned" into larger chunks such as large AI/ML datasets – WD believes there are opportunities to organize workloads for better performance and efficiencies with lower TCO by accelerating the adoption of zoned storage devices.

Zoned Storage is an open-source, standards-based initiative to enable data centers to scale efficiently for the zettabyte storage capacity era. It involves the ability to store and retrieve information using shingled magnetic recording (SMR) in hard disk drives (HDDs) to increase the storage density and its companion technology called Zoned Name Spaces in solid state drives (SSDs).

In a nutshell, Zoned Storage Devices are block storage devices that have their address space divided into zones. ZSDs impose unconventional writing rules: zones can only be written sequentially and starting from the beginning of the zone. In addition, data within a zone cannot be arbitrarily overwritten.

The only way to overwrite an already written zone is to reset the zone write pointer, effectively deleting all the data in the Zone, and to restart writing from the beginning of the zone. Reading data, on the other hand, is largely unrestricted and the data can be read in the same manner as on traditional storage devices.

The Zoned Storage Device concept is standardized for storage devices as:

• ZBC: Zoned Block Commands in T10/SAS
• ZAC: Zoned Device ATA Command Set in T13/SATA
• ZNS: Zoned Namespaces in NVMe (Technical Proposal in progress)
  

The Shingled Magnetic Recording (SMR) technology has been introduced in Hard Disk Drives (HDDs) to enable increased areal density and larger capacities and cost-effectiveness for HDDs. In SMR, unlike conventional recording, tracks are written in an overlapping manner. This allows tracks to be more tightly packed and hence achieves a higher density of recording.

However, once the tracks are overlapped, they cannot be written independently. In order to manage the recording, the disk surface is divided into Zones with a gap left between Zones. This allows each Zone to be written and erased independently. Multiple approaches are possible to manage the recording restriction. The traditional approach assumes the device handles the recording constraint internally and expose a conventional interface to the host. Yet for large-scale systems, where performance and space utilization must be carefully managed, there are disadvantages for device-side localized management. Managing the complexity on the host side is almost a requirement for large storage systems and is the go-to choice for hyperscale data centers.

Host Managed SMR is standardized in INCITS T10/T13 as Zoned Block Commands (ZBC) and Zoned Device ATA Command Set (ZAC) for SAS and SATA respectively.

Zoned Name Spaces is an extension of the NVMe standard. This technology allows us to offer SSDs that are zoned similarly to HDDs, thereby allowing all storage, whether it be HDD or SSD, to be seen as though it were one technology.

ZNS NVMe SSDs offer several benefits over traditional NVMe SSDs. Conforming to the ZNS feature set, as defined and governed by NVM Express, Inc., these ZNS SSDs are designed to lower write amplification, increase capacity, and provide improved throughput and latency. ZNS SSDs achieve these improvements by aligning "zones" to the internal physical properties of the SSDs, eliminating inefficiencies in the placement of data.

For Solid-State Drives (SSDs), the restriction of having regions that can only be written sequentially and need to be erased before new data can be written to it is an inherent property of how flash storage operates.

When SSDs were initially introduced, they implemented an internal management system, called Flash Translation Layer (FTL), that would manage this recording constraint. The FTL allowed SSDs to be used in place of HDDs without the need to immediately rewrite the storage software stack. The local management inside the SSD creates some inefficiencies, however. In particular, there is:

• Write Amplification: To manage the constraints of not overwriting data, the SSD has to internally move data around to reclaim unused data locations, this is called Garbage Collection (GC). The GC process causes multiple writes of the same data (hence the term Write Amplification) creating further wear to the Flash media and reducing the life of the SSD.
• Over-Provisioning: Extra space (in some cases as large as 28%) has to be reserved for moving data around for garbage collection and to improve its efficiency.
• Large DRAM requirement: To maintain the FTL logical-to-physical mapping, a large amount of DRAM is needed in the device. Note that the amount of DRAM required increases with the size of the device.
• QoS variability due to the garbage collection kicking in at any moment without explicit control from the host software.

These inefficiencies were acceptable when SSDs were first introduced because the software stacks and interfaces were designed for HDDs, and their response times were much larger. The enhancement in performance constituted a good trade-off.

With time, however, the industry has moved to create more efficient interfaces and software stacks that can take advantage of the inherently lower latency and higher bandwidth of Flash. In particular, the Non-Volatile Memory Express (NVMe) interface specification was created, and the corresponding low overhead software stack developed.

The status in the industry today is that the local optimizations that have evolved in the SSD FTL have now become detrimental to at-scale data infrastructure deployments; in particular, a desire has emerged from many customers to enable a managed sequential workload align with Flash erase blocks. In response to this, the NVMe organization is standardizing Zoned Namespaces (ZNS) that will allow the host to direct I/Os to separate workloads, and improves latency, throughput and costs efficiency by moving the bulk of management to the host.

"There's no doubt that the industry needs to come together and find new and creative ways to design and architect infrastructures for the future to more intelligently and efficiently optimize workloads, and storage and compute infrastructure," said Ashish Nadkarni, group vice president, Infrastructure Systems, Platforms and Technologies at IDC. "With total HDD and SSD revenue expected to reach 27 percent of total IT infrastructure spend by 2022, we see SMR, SSDs and ZNS together as a standout opportunity for cloud and hyperscale data center customers to optimize infrastructure for better cost economies at scale. Unifying the ZNS and SMR architectures via open, standards-based initiatives is a logical industry step, which can take advantage of SMR HDD areal density gains and new innovations in Flash. Whoever enters the market first, will definitely have a competitive advantage on TCO and the learning curve."

Cloud service providers are investing in host software that can manage high-capacity SMR HDDs to deliver better TCO to their customers at scale. WD's 15TB Ultrastar DC HC620 SMR HDDs are shipping in volume, with a technology demonstration of its 20TB SMR HDD at a press event and private customer gathering being held in Beijing this week.

Data Infrastructure – Adopting Zoned Storage

Zoned Block Devices deliver at-scale efficiencies, yet to take advantage of these new devices requires a heavy lift as they are not backward compatible. While traditional storage devices have no restriction on writes, Zoned Block Devices have the restriction that writes within a Zone must be sequential. The main consequence of this is that the software stack must be updated.

The first component that needs to be updated is the operating system. This is not a trivial task in modern multi-tasking operating systems running on multi-core & multi-socket servers, typical of data-center deployments. The Linux community has applied substantial effort to support Zone Block Devices in general, and SMR in particular.

In addition to the Linux kernel support, there are a number of utilities and applications that support Zone Block Devices, such as fio, blktests and util-linux.

Western Digital has launched ZonedStorage.io. Working with the open-source community, customers and industry players, ZonedStorage.io is aimed to jump-start application development initiatives within the developer community as well as help data center infrastructure engineers take advantage of Zoned Storage technologies. The site hosts information about software development libraries, tools, ZNS information, SMR resources and reference architectures that provide a unified framework to manage purpose-built ZNS SSDs and capacity-optimized SMR HDDs.

Western Digital's suite of products consists of the Ultrastar ZNS NVMe development SSD and the Ultrastar DC HC620 15TB SMR HDD. The 15TB Ultrastar DC HC620 is currently available. WD says that the Ultrastar ZNS NVMe development SSD is available to select customers.