Udemy - Deep Learning for NLP - Part 7

Deep Learning for NLP - Part 7



MP4 | Video: h264, 1280x720 | Audio: AAC, 44.1 KHz, 2 Ch
Genre: eLearning | Language: English + srt | Duration: 36 lectures (6h 4m) | Size: 2.67 GB
Model Compression for NLP
What you'll learn:
Deep Learning for Natural Language Processing
Model Compression for NLP
Pruning
Quantization
Knowledge Distillation
Parameter sharing
Matrix decomposition
DL for NLP

Requirements
Basics of machine learning
Basic understanding of Transformer based models and word embeddings
Transformer Models like BERT and GPT

Description
In recent years, the fields of natural language processing (NLP) and information retrieval (IR) have made tremendous progress thanks to deep learning models like Recurrent Neural Networks (RNNs), Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTMs) networks, and Transformer based models like Bidirectional Encoder Representations from Transformers (BERT), Generative Pre-training Transformer (GPT-2), Multi-task Deep Neural Network (MT-DNN), Extra-Long Network (XLNet), Text-to-text transfer transformer (T5), T-NLG and GShard.

These models are humongous in size: BERT (340M parameters), GPT-2 (1.5B parameters), T5 (11B parameters, 21.7GB), etc. On the other hand, real world applications demand small model size, low response times and low computational power wattage. In this course, we discuss five different types of methods (Pruning, Quantization, Knowledge Distillation, Parameter Sharing, Tensor Decomposition) for compression of such models to enable their deployment in real industry NLP projects. Given the critical need of building applications with efficient and small models, and the large amount of recently published work in this area, we believe that this course organizes the plethora of work done by the "deep learning for NLP" community in the past few years and presents it as a coherent story.