Next year we're going to see the majority of new products coming out with NVM Express (NVMe), with products using the NVM Express Over Fabrics (NVMe-oF) specification and the the NVM Express Management Interface (NVMe-MI).
The NVMe interface has unlocked the internal performance of flash in SSDs, which previously were hampered by architectures designed for spinning disk. In 2019, clients and enterprise applications will increasingly use NVMe to take advantage of the not only the performance of flash, but other memory-class storage such as 3D Xpoint and other emerging options.
In addition, the NVMe-oF will allow for older storage technologies to be woven in where they still make sense. Officially released in May 2016,, the NVMe-oF standard is designed to enable nonvolatile memory express message-based commands to transfer data between a host computer and a target solid-state storage device or system over a network. Benefits include increased performance and reduced network latency and bottlenecks.
The NVMe-oF specification supports the fabric of choice, whether it’s Ethernet and Omnipass, among others, to take advantage of NVMe end-to-end using a tunneling protocol. It enables the connection of thousands of devices in a data center.
Version 1.1 of NVMe-oF, coming out early next year, will include a Transmission Control Protocol (TCP) layer, which is in addition to Remote Direct Memory Access (RDMA) and fiber channel that allowed for use of InfiniBand, Ethernet or Omnipass. This will allow vendors that have existing investments in network interface cards without RDMA capability can take advantage of NVMe-oF.
The benefits for developers is that it gives them the ability to migrate to an NVMe technology and migrate away from Internet Small Computer Systems Interface (iSCSI). NVMe-oF/TCP is an option for enterprises who want to leverage their Ethernet infrastructure and avoid the complexities of Remote Direct Memory Access (RDMA) protocol.
Many industry leaders including Facebook, Google, Dell EMC, Intel, and others are standing behind the NVMe/TCP standard.
The NVMe standard also allows to move management away from the SSDs and put it on the host using the NVM Express Management Interface (NVMe-MI). This comes as the concept of computational storage is gaining ground, which is when processing power is put on the storage devices themselves.
In addition, NVMe 1.4 is coming mid next year. It will add IO determinism to guarantee tight and consistent latency across networking and storage. It will enable hosts to treat an SSD as many small sub-SSDs and process IO in parallel in each small sub-SSD. Meanwhile, the addition of the management specification provides the capability of managing device enclosures.
The NVMe standard was initially designed with flash in mind, but it is open to accommodating new technologies such as emerging storage-class memories, such as 3D Xpoint and Optane. The SD Card Association is adopting NVMe to scale performance as they move forward with SD 7.0.
Micron Technology has already used NVMe-oF in its SolidScale architecture. It leverages SSD coupled with NVMe and NVMe over Fabric to scale at near linear performance rates. The architecture was created for low-latency, high-performance access to compute and storage resources and specifically address CPU underutilization in the data center as NVMe SSDs deployed in application servers at the time were on average using less than 50% of their IOPS and capacity.
The company sees the enterprise and cloud customer bases as they key adopters.
Despite the commercialization of 3D Xpoint and Intel’s recent Optane push, Micron is going to stick with its NAND-based NVMe product lines for storage solutions, while 3D Xpoint will be treated more like a memory rather another candidate for the NVMe interface. Getting the data closer to the processor would make sense for deep learning and machine learning type algorithms, where you have a particular data set that wants to stay in a cache, according to Micron. But the company is not looking to do storage class memory on an NVME bus.