Windows 8 To Efficiently Support Large Capacity Disks
Microsoft's Windows 8 operating system will be designed t fully take advantage of hard disk drives larger then 2.2TB, by supporting both types of Advanced Format (AF) disks - 512e and 4K Native.
According to Bryan Matthew, a program manager on Windows 8's Storage & File System team, AF disks have been a key design point for new features and technologies in Windows 8; as a result, Windows 8 is the first OS with full support for both types of AF disks - 512e and 4K Native.
Background
All hard disk drives include some form of built-in error correction information and logic - this enables hard disk drive vendors to automatically deal with the Signal-to-Noise Ratio (SNR) when reading from the disk platters. As disk capacity increases, bits on the disk get packed closer and closer together; and as they do, the SNR of reading from the disk decreases. To compensate for decreasing SNR, individual sectors on the disk need to store more Error Correction Codes (ECC) to help compensate for errors in reading the sector. Modern disks are now at the point where the current method of storing ECCs is no longer an efficient use of space, - that is, a lot of the space in the current 512-byte sector is being used to store ECC information instead of being available for you to store your data. This, among other things, has led to the introduction of larger sector sizes.
With a larger sector size, a different scheme can be used to encode the ECC; this is more efficient at correcting for errors, and uses less space overall. This efficiency helps to enable even larger capacities for the future. Hard disk manufacturers agreed to use a sector size of 4KB, which they call "Advanced Format (AF)," and they introduced the first AF drive to the market in late 2009. Since then, hard disk manufacturers have rapidly transitioned their product lines to AF media, with the expectation that all future storage devices will use this format.
With an AF disk, the layout of data on the media is physically arranged in 4KB blocks. Updates to the media can only occur at that granularity, and so, to enable logical block addressing in smaller units, the disk needs to do some special work. Writes done in units of the physical sector size do not need this special work, so you can think of the physical sector size as the unit of atomicity for the media.
As illustrated below, a 4KB physical sector can be logically addressed with 512-byte logical sectors. In order to write to a single logical sector, the disk cannot simply move the disk head over that section of the physical sector and start writing. Instead, it needs to read the entire 4KB physical sector into a cache, modify the 512-byte logical sector in the cache, and then write the entire 4KB physical sector back to the media (replacing the old block). This is called Read-Modify-Write.
Disks with this emulation layer to support unaligned writes are called 4K with 512-byte emulation, or "512e" for short. Disks without this emulation layer are called "4K Native."
As a result of Read-Modify-Write, performance can potentially suffer in applications and workloads that issue large amounts of unaligned writes. To provide support for this type of media, Windows needs to ensure that applications can retrieve the physical sector size of the device, and applications (both Windows applications and 3rd party applications) need to ensure that they align I/O to the reported physical sector size.
The Windows 8 approach
In order to deal with these potential issues, Microsoft introduced a new and enhanced existing API to better enable applications to query for the physical sector size of a disk. The company enhanced large-sector awareness within the NTFS file system, including ensuring appropriate sector padding when performing extending writes (writing to the end of the file). Microsoft's team also incorporated large-sector awareness in the new VHDx file format used by Hyper-V to fully support both types of AF disks. Last but not least, the team enhanced the Windows boot code to work correctly when booting from 4K native disks.
Microsoft is also working with other product teams within Microsoft and across the industry (e.g. database application developers) to ensure efficient and correct behavior with AF disks.
Disk partitioning and UEFI
Beginning with Windows Vista 64-bit, Windows has supported the ability to boot from a GPT partitioned hard disk drive with one key requirement - the system firmware must be UEFI. The GUID Partition Table (GPT) is a scalable partitioning scheme that allows for up to 64-bits of information to store the number that represents the maximum size of a disk, which in turn allows for up to a theoretical maximum of 9.4 ZettaByte. GTP is part of the Unified Extensible Firmware Interface (UEFI) specification.
The UEFI specification was developed as a replacement for BIOS. It was designed from the ground up to work with very large capacity drives by utilizing the GUID partition table, or GPT - although some BIOS implementations have attempted to prolong their own relevance and utility by using workarounds for large capacity drives (e.g. a hybrid MBR-GPT partitioning scheme). These mechanisms can be quite fragile, and can place data at risk. Therefore, Windows has consistently required modern UEFI firmware to be used in conjunction with the GPT scheme for boot disks.
According to Bryan Matthew, beginning with Windows 8, multiple new capabilities within Windows will necessitate UEFI. The combination of UEFI firmware + GPT partitioning + LBA allows Windows to fully address very large capacity disks with ease.
Windows 8 based systems will use UEFI to help enable new features and scenarios (e.g. Secure Boot, Encrypted Drive, and Fast Start-up). Microsoft promised that when Windows 8 is released, new systems will support installing Windows 8 to, and booting from, a 3TB or bigger disk.
Background
All hard disk drives include some form of built-in error correction information and logic - this enables hard disk drive vendors to automatically deal with the Signal-to-Noise Ratio (SNR) when reading from the disk platters. As disk capacity increases, bits on the disk get packed closer and closer together; and as they do, the SNR of reading from the disk decreases. To compensate for decreasing SNR, individual sectors on the disk need to store more Error Correction Codes (ECC) to help compensate for errors in reading the sector. Modern disks are now at the point where the current method of storing ECCs is no longer an efficient use of space, - that is, a lot of the space in the current 512-byte sector is being used to store ECC information instead of being available for you to store your data. This, among other things, has led to the introduction of larger sector sizes.
With a larger sector size, a different scheme can be used to encode the ECC; this is more efficient at correcting for errors, and uses less space overall. This efficiency helps to enable even larger capacities for the future. Hard disk manufacturers agreed to use a sector size of 4KB, which they call "Advanced Format (AF)," and they introduced the first AF drive to the market in late 2009. Since then, hard disk manufacturers have rapidly transitioned their product lines to AF media, with the expectation that all future storage devices will use this format.
With an AF disk, the layout of data on the media is physically arranged in 4KB blocks. Updates to the media can only occur at that granularity, and so, to enable logical block addressing in smaller units, the disk needs to do some special work. Writes done in units of the physical sector size do not need this special work, so you can think of the physical sector size as the unit of atomicity for the media.
As illustrated below, a 4KB physical sector can be logically addressed with 512-byte logical sectors. In order to write to a single logical sector, the disk cannot simply move the disk head over that section of the physical sector and start writing. Instead, it needs to read the entire 4KB physical sector into a cache, modify the 512-byte logical sector in the cache, and then write the entire 4KB physical sector back to the media (replacing the old block). This is called Read-Modify-Write.
Disks with this emulation layer to support unaligned writes are called 4K with 512-byte emulation, or "512e" for short. Disks without this emulation layer are called "4K Native."
As a result of Read-Modify-Write, performance can potentially suffer in applications and workloads that issue large amounts of unaligned writes. To provide support for this type of media, Windows needs to ensure that applications can retrieve the physical sector size of the device, and applications (both Windows applications and 3rd party applications) need to ensure that they align I/O to the reported physical sector size.
The Windows 8 approach
In order to deal with these potential issues, Microsoft introduced a new and enhanced existing API to better enable applications to query for the physical sector size of a disk. The company enhanced large-sector awareness within the NTFS file system, including ensuring appropriate sector padding when performing extending writes (writing to the end of the file). Microsoft's team also incorporated large-sector awareness in the new VHDx file format used by Hyper-V to fully support both types of AF disks. Last but not least, the team enhanced the Windows boot code to work correctly when booting from 4K native disks.
Microsoft is also working with other product teams within Microsoft and across the industry (e.g. database application developers) to ensure efficient and correct behavior with AF disks.
Disk partitioning and UEFI
Beginning with Windows Vista 64-bit, Windows has supported the ability to boot from a GPT partitioned hard disk drive with one key requirement - the system firmware must be UEFI. The GUID Partition Table (GPT) is a scalable partitioning scheme that allows for up to 64-bits of information to store the number that represents the maximum size of a disk, which in turn allows for up to a theoretical maximum of 9.4 ZettaByte. GTP is part of the Unified Extensible Firmware Interface (UEFI) specification.
The UEFI specification was developed as a replacement for BIOS. It was designed from the ground up to work with very large capacity drives by utilizing the GUID partition table, or GPT - although some BIOS implementations have attempted to prolong their own relevance and utility by using workarounds for large capacity drives (e.g. a hybrid MBR-GPT partitioning scheme). These mechanisms can be quite fragile, and can place data at risk. Therefore, Windows has consistently required modern UEFI firmware to be used in conjunction with the GPT scheme for boot disks.
According to Bryan Matthew, beginning with Windows 8, multiple new capabilities within Windows will necessitate UEFI. The combination of UEFI firmware + GPT partitioning + LBA allows Windows to fully address very large capacity disks with ease.
Windows 8 based systems will use UEFI to help enable new features and scenarios (e.g. Secure Boot, Encrypted Drive, and Fast Start-up). Microsoft promised that when Windows 8 is released, new systems will support installing Windows 8 to, and booting from, a 3TB or bigger disk.
