Intel's nGraph Compiler, a framework-neutral deep neural network (DNN) model compiler, is now open source, allowing support for multiple deep learning frameworks while optimizing models for multiple hardware solutions.
"Finding the right technology for AI solutions can be daunting for companies, and it's our goal to make it as easy as possible. With the nGraph Compiler, data scientists can create deep learning models without having to think about how that model needs to be adjusted across different frameworks, and its open source nature means getting access to the tools they need, quickly and easily," said Arjun Bansal, VP, AI Software, Intel.
With nGraph, data scientists can focus on data science rather than worrying about how to adapt their DNN models to train and run efficiently on different devices.
Currently, the nGraph Compiler supports three deep learning compute devices and six third-party deep learning frameworks: TensorFlow, MXNet, neon, PyTorch, CNTK and Caffe2. Users can run these frameworks on several devices: Intel Architecture (x86, Intel Xeon and Xeon Phi), GPU (NVIDIA cuDNN), and Intel Nervana Neural Network Processor (NNP).
When Deep Learning (DL) frameworks first emerged as the vehicle for running training and inference models, they were designed around kernels optimized for a particular device. As a result, many device details were being exposed in the model definitions, complicating the adaptability and portability of DL models to other, or more advanced, devices.
The traditional approach means that an algorithm developer faces tediousness in taking their model to an upgraded device. Enabling a model to run on a different framework is also problematic because the developer must separate the essence of the model from the performance adjustments made for the device, translate to similar ops in the new framework, and finally make the necessary changes for the preferred device configuration on the new framework.
Intel designed the nGraph library to reduce these kinds of engineering complexities. While optimized kernels for DL primitives are provided through the project and via libraries like Intel Math Kernel Library for Deep Neural Networks (Intel MKL-DNN), there are also several compiler-inspired ways in which performance can be further optimized.