Backblaze Statistics Unveil Higher-Than-Typical Failure Rate Among Some 12TB Seagate HDDs
Backblaze, a company that operates about 123,000 hard disk drives in its datacenters, has identified a higher-than-typical failure rate among some of its 12TB Seagate drives.
The company typically releases reports that includes the hard drive failure rates for the data drive models in operation in its data centers.
For their evaluation, Backblaze remove from consideration those drives that were used for testing purposes and those drive models for which theh company did not have at least 5,000 drive days during Q4.
In their report for 2019, the company says that the 12TB Seagate drives showed a failure rate of 3,32%, which is among the higher in the company's total HDD fleet from various brands. Backblaze had 37,004 Seagate Exos ST12000NM0007 HDDs installed, and 1,156 of those drives failed. Those HDD had logged 12.721.076 drive days.
Yev Pusin, director of marketing, cloud storage and backup at Backblaze, Inc. said:
"It’s customary for HDD manufacturers like Seagate, when working with data centers and cloud service providers, to ensure successful deployment of large-scale drive fleets, and as such we’re working closely with them to analyze the drives and their performance. This analysis usually includes things like testing new drive platforms in real workload environments, providing telemetry tools to predict failures, performing ongoing custom adjustments, and employing firmware development and replacement units (RMAs).
Customer data durability is paramount for both Backblaze and Seagate, so as we analyze root causes and implications we’re also working together on a migration effort to replace these particular drives in our data centers. In the short term, failure rates for a subset of our drives may increase, but we have processes in place to adjust for that fluctuation.
In the near term, we expect to see moderately increased failure rates for this specific subset of 12TB drives, but as we complete the drive migration, we project our fleet’s failure rates will restore to historical norms."
The report also showed that the only drive model that did not to have a failure during 2019 was the 4 TB Toshiba, model: MD04ABA400V. Although that sounds impressive, the data sample is still somewhat small. For example, if there had been just 1 (one) drive failure during the year, the Annualized Failure Rate (AFR) for that Toshiba model would be 0.92%—still excellent, not 0%.
The Toshiba 14 TB drive, model MG07ACA14TA, also seems to perform very well at a 0.65% AFR, similar to the rates put up by the HGST drives. For their part, the Seagate 6 TB and 10 TB drive continue to be solid performers with annualized failure rates of 0.96% and 1.00% respectively.
The AFR for 2019 for all drive models was 1.89% which is much higher than 2018. About 75% of the different drive models experienced a rise in AFR from 2018 to 2019.
According to Backblaze, there are two primary drivers behind this rise.
First, the 8 TB drives as a group seem to be having a mid-life crisis as they get older, with each model exhibiting their highest failure rates recorded. While none of the rates is cause for worry, they contribute roughly one fourth (1/4) of the drive days to the total, so any rise in their failure rate will affect the total. The second factor is the Seagate 12 TB drives.
In Q4 2019, Backblaze also started qualifying Seagate 16 TB drives, model: ST16000NM001G. As of the end of Q4 the company had 40 (forty) drives in operation, with a total of 1,440 drive days—well below Backblaze's 5,000 drive day threshold for Q4, so they didn’t make the 2019 chart. There have been 0 (zero) failures through Q4, making the AFR 0%, a good start for any drive.
In Q4 2019 there were 20 (twenty) Toshiba 8 TB drives, model: HDWF180. These drives have been installed for nearly two years. In Q4, they only had 1,840 drive days, below the reporting threshold, but lifetime they do have 13,994 drive days with only 1 drive failure, giving us an AFR of 2.6%.
Backblaze has also 20 (twenty) HGST 10 TB drives, model: HUH721010ALE600 in the operation. These drives have been in service a little over one year. They reside in the same Backblaze Vault as the Seagate 10 TB drives. The HGST drives recorded only 1,840 drive days in Q4 and a total of 8,042 since being installed. There have been 0 (zero) failures.
In addition, the company added in Q1 202020 (twenty) Toshiba 16 TB drives, model: MG08ACA16TA. They have logged a total of 100 drive days, so it was way too early to say anything other than more to come in the Q1 2020 report.
The complete data set used to create the report is available on this page.