Friday, October 20, 2017
Search
  
Submit your own News for
inclusion in our Site.
Click here...
Breaking News
Intel Capital Announces $60 million of New Investments in Data-Focused Startups
Google Offers Bug Bounty for Mobile Apps
Nintendo Switch Update Brings Video Capture and More
Intel Health Application Platform Helps Health Care Providers Reduce Hospital Readmissions
Razer Electra V2 Gaming Headsets Bring Virtual Surround Sound and Aluminum Headband to Everyone
Alphabet Invests $1 Billion to Lyft
Microsoft TruePlay Anti-cheat Game Feature to be Added to UWP Games
Apple Cut iPhone 8 Production by 50 percent: report
Active Discussions
Which of these DVD media are the best, most durable?
How to back up a PS2 DL game
Copy a protected DVD?
roxio issues with xp pro
Help make DVDInfoPro better with dvdinfomantis!!!
menu making
Optiarc AD-7260S review
cdrw trouble
 Home > News > General Computing > IBM Cla...
Last 7 Days News : SU MO TU WE TH FR SA All News

Friday, August 11, 2017
IBM Claims Record Deep Learning Performance


IBM Research has reported an algorithmic breakthrough for deep learning that comes close to achieving the holy grail of ideal scaling efficiency

IBM's new distributed deep-learning (DDL) software enables a nearly linear speedup with each added processor. The development is intended to achieve similar speedups for each server added to IBM's DDL algorithm.

The company published in arXiv close to ideal scaling with new distributed deep learning software which achieved record communication overhead and 95% scaling efficiency on the Caffe deep learning framework over 256 NVIDIA GPUs in 64 IBM Power systems.

Previous best scaling was demonstrated by Facebook AI Research of 89% for a training run on Caffe2, at higher communication overhead. IBM Research also beat Facebook's time by training the model in 50 minutes, versus the 1 hour Facebook took. Using this software, IBM Research achieved a new image recognition accuracy of 33.8% for a neural network trained on a very large data set (7.5M images). The previous record published by Microsoft demonstrated 29.8% accuracy.

A technical preview of this IBM Research Distributed Deep Learning code is available today in IBM PowerAI 4.0 distribution for TensorFlow and Caffe.

Deep learning is a widely used AI method to help computers understand and extract meaning from images and sounds through which humans experience much of the world. It holds promise to fuel breakthroughs in everything from consumer mobile app experiences to medical imaging diagnostics. But progress in accuracy and the practicality of deploying deep learning at scale is gated by technical challenges, such as the need to run massive and complex deep learning based AI models - a process for which training times are measured in days and weeks.

IBM Research has been focused on reducing these training times for large models with large data sets. The objective is to reduce the wait-time associated with deep learning training from days or hours to minutes or seconds, and enable improved accuracy of these AI models. To achieve this, IBM's reseachers are tackling grand-challenge scale issues in distributing deep learning across large numbers of servers and NVIDIA GPUs.

Most popular deep learning frameworks scale to multiple GPUs in a server, but not to multiple servers with GPUs. Specifically, IBM's team (Minsik Cho, Uli Finkler, David Kung, Sameer Kumar, David Kung, Vaibhav Saxena, Dheeraj Sreedhar) wrote software and algorithms that automate and optimize the parallelization of this very large and complex computing task across hundreds of GPU accelerators attached to dozens of servers.

The software does deep learning training fully synchronously with very low communication overhead. As a result, when the researchers scaled to a large cluster with 100s of NVIDIA GPUs, it yielded record image recognition accuracy of 33.8% on 7.5M images from the ImageNet-22k dataset vs the previous best published result of 29.8% by Microsoft. A 4% increase in accuracy is a big leap forward; typical improvements in the past have been less than 1%. IBM's distributed deep learning (DDL) approach enabled the researchers to not just improve accuracy, but also to train a ResNet-101 neural network model in just 7 hours, by leveraging the power of 10s of servers, equipped with 100s of NVIDIA GPUs; Microsoft took 10 days to train the same model. This achievement required we create the DDL code and algorithms to overcome issues inherent to scaling these otherwise powerful deep learning frameworks.

The company is making its DDL suite available free to any PowerAI platform user.



Previous
Next
HTC Continues to Report Losses        All News        HBO Offered $250,000 to Hackers
Sharp Sees Collaboration With Japan Display     General Computing News      HBO Offered $250,000 to Hackers

Get RSS feed Easy Print E-Mail this Message

Related News
IBM Unveils New High-Powered Analytics System for Fast Access to Data Science
New Sony Magnetic Tape Storage Technology Supports High-Capacity 330 TB Recording
IBM Z Mainframe Features Pervasive Data Encryption
Sony's "Core Libraries" of Deep Learning Tools are now Open source
Researchers Say Silicon Nanosheets is the Path to 5nm Transistors
Wanda and IBM To Bring IBM Cloud to China
Researchers Store Data on A Single Atom
IBM Building First Universal Quantum Computers
IBM tops U.S. Patents list Again , Samsung Follows
LG Uses 'Deep Learning' Technology To Make Home Appliances Smarter
TSMC And IBM Detail Their 7nm Progress At 2016 IEDM
IBM and NVIDIA Team Up on New Platform For Deep Learning

Most Popular News
 
Home | News | All News | Reviews | Articles | Guides | Download | Expert Area | Forum | Site Info
Site best viewed at 1024x768+ - CDRINFO.COM 1998-2017 - All rights reserved -
Privacy policy - Contact Us .