Full-Stack, GPU-based Acceleration of Deep Learning

Nvidia

Summit 447

Summary

This tutorial focuses on describing techniques to allow deep learning practitioners to accelerate the training and inference of large deep networks while also reducing memory requirements across a spectrum of off-the-shelf hardware for important applications such as autonomous driving and large language models. Topics include, but are not limited to:
  • Deep learning specialized hardware overview. We will review the architecture of most commonly used deep learning acceleration hardware: GPU and TPU. We will cover main computational processors and memory modules.
  • How deep learning is performed on this hardware. We will cover aspects of algorithmic intensity and provide overview of theoretical aspects of compute. Attendees will learn how to estimate processing time and latency by looking only at hardware specs and the network architecture.
  • Best practices for acceleration. We will provide an overview of best practices to design efficient neural networks. Topics of interest will include guidance for channel number selection, compute heavy operations, reduction operations etc.
  • Existing tools for model acceleration. In this part we will focus on existing tools to accelerate a trained neural network on GPU devices. Particularly, we will discuss operation folding, TensorRT, ONNX graph optimization, sparsity.
  • Research overview of recent techniques. This part will cover most recent advanced techniques for post training model optimization with the focus on most recent works (past 4 years). Range of topics will include pruning, quantization, model distillation, NAS etc.
  • Foundation models. This part will cover most recent advanced techniques for training and deploying foun- dation models efficiently.

Schedule

13:3013:35Opening Remarks
13:3514:30 Jason Clemons Foundations of DL Hardware And How to Apply Them.
14:3515:30 Maying Shen Neural Network Acceleration.
15:0015:30Coffee Break
15:3016:30 Hongxu (Danny) YinEfficient Vision Language Models.
16:3016:35Closing Remarks

Instructors

Maying Shen is currently a senior autonomous driving research engineer at NVIDIA. Prior to joining NVIDIA, she graduated from CMU majoring in computer vision, where she developed her interest in seeing the world through the computer's eyes. Her interests include deep learning efficiency from both, training and inference side, working on aspects such as neural network pruning, distillation, or quantization among others.
Jason Clemons received his Ph.D. in computer science and engineering from the University of Michigan, Ann Arbor, MI, USA where he researched computer architectures for mobile computer vision. In his senior research scientist role at NVIDIA his current research focuses on domain-specific computing, in particular the intersection of machine learning, computer vision, and computer architecture. He has worked on machine learning accelerators, computer vision accelerators, accelerating DNN training on GPUs, and accelerating RL using GPUs. He is an IEEE senior member and serves on IEEE International Symposium on Performance Analysis of Systems and Software steering committee.
Hongxu (Danny) Yin is a senior research scientist at Learning and Perception Research (LPR) at NVIDIA. He obtained his Ph.D. at Princeton University, New Jersey, USA, and B. Eng. from Nanyang Technological University, Singapore. He is a recipient of Princeton Yan Huo 94* Graduate Fellowship, Princeton Best Dissertation Finalist within Department, Princeton Natural Sciences and Engineering Fellowship, Defense Science & Technology Agency gold medal, and Thomson Asia Pacific Holdings gold medal. His research interests mainly include data-/execution-efficient and secure deep learning overseeing CNNs and transformers. He has been the organizer of several tutorial/workshop at CVPR and ICCV. He has been featured as Global Outstanding Chinese Power 100 Award by 36Kr and Top 60 Elite Chinese in North America by Forbes.

Organizers

  • Maying Shen, Senior Research Engineer
  • Jason Clemons, Senior Research Scientist
  • Hongxu (Danny) Yin, Senior Research Scientist
  • Pavlo Molchanov, Principal Research Scientist
  • Jose M. Alvarez, Director, Applied research
  • Jan Kautz, VP of research