New levels of accuracy in computer vision, from image recognition and detection, to generating images with GANs, have been achieved by increasing the size of trained models. Fast turn-around times while iterating on the design of such models would greatly improve the rate of progress in this new era of computer vision.
This tutorial will describe techniques that utilize half-precision floating point representations to allow deep learning practitioners to accelerate the training of large deep networks while also reducing memory requirements.
The talks and sessions below will provide a deep-dive into available software packages that enable easy conversion of models to mixed precision training, practical application examples, tricks of the trade (mixed precision arithmetic, loss scaling, etc.), as well as considerations relevant to training many popular models in commonly used deep learning frameworks including PyTorch and TensorFlow.
Topic | Speaker | ||
---|---|---|---|
Introduction | video | Arun Mallya | |
Basics and Fundamentals | |||
Training Neural Networks with Tensor Cores | video | Dusan Stosic | |
Code Optimization Tricks | |||
PyTorch Performance Tuning Guide | video | Szymon Migacz | |
Application Case Studies | |||
Mixed Precision Training for Conditional GANs | video | Ming-Yu Liu | |
Mixed Precision Training for FAZE: Few-shot Adaptive Gaze Estimation | video | Shalini De Mello | |
Mixed Precision Training for Video Synthesis | video | Ting-Chun Wang | |
Mixed Precision Training for Convolutional Tensor-Train LSTM | video | Wonmin Byeon | |
Mixed Precision Training for 3D Medical Image Analysis | video | Dong Yang |
Dusan Stosic | Paulius Micikevicius | Szymon Migacz | Ming-Yu Liu | |||
Shalini De Mello | Ting-Chun Wang | Wonmin Byeon | Dong Yang | |||
Arun Mallya |