فیلم و فایل‌های دوره آموزشی یادگیری ژرف | مدرس : دکتر محمد علی کیوان راد

Description

Course
Materials

Lecture 1

Course Introduction

·        
Recent success with Deep Learning

·        
Historical context

·        
Course plan

Suggested
Readings:

۱٫    
[Linear
Algebra Review]

۲٫    
[Probability
Review]

۳٫    
 [Optimization (SGD) Review]

[python tutorial] [slides]

Lecture 2

A Review of Artificial
Neural Networks 1

·        
Perceptron

·        
Stochastic Gradient Descent

·        
Backpropagation

Suggested
Readings:

۱٫    
[Classification
and Linear Classifiers]

۲٫    
[Backpropagation]

[slides] [video]

Lecture 3

A Review of Artificial
Neural Networks 2

·        
Activation Functions (Sigmoid, Tanh, ReLU, Leaky ReLU, ELU, Maxout)

·        
Neural Network architectures

Suggested
Readings:

۱٫    [Neural
Nets notes 1]

[slides] [video]

Lecture 4

A Review of Artificial
Neural Networks 3

·        
Data Preprocessing

·        
Weight Initialization

·        
Batch Normalization

Suggested
Readings:

۱٫    [Neural
Nets notes 2]

۲٫    [Batch
normalization]

[slides] [video]

Lecture 5

A Review of Artificial
Neural Networks 4

·        
Model Ensembles

·        
Regularization

·        
Dropout

·        
Regularization: A common pattern

Suggested
Readings:

۱٫    [Neural
Nets notes 2]

۲٫    [Neural
Nets notes 3]

۳٫    [Dropout]

۴٫    [Regularization]

[slides] [video]

Lecture 6

A Review of Artificial
Neural Networks 5

·        
Hyperparameter Optimization

·        
Loss functions

·        
Parameter updates

·        
Transfer Learning

Suggested
Readings:

۱٫    [Neural
Nets notes 2]

۲٫    [Neural
Nets notes 3]

۳٫    [linear
classify]

۴٫    [intro-to-cross-entropy-loss]

۵٫    [transfer-learning]

[slides] [video]

Lecture 7

A quick review of
Probabilistic Graphical Models

·        
Probabilistic Graphical Models

·        
Bayesian Networks

·        
Dynamic Bayesian Networks

·        
Markov Random Fields

Suggested
Readings:

۱٫    [Pattern
Recognition and Machine Learning]

۲٫    [Probabilistic
graphical models: principles and techniques]

۳٫    [Dynamic
Bayesian networks]

۴٫    [PGM coursera]

۵٫    [PGM AUT: CL1,
CL2, CL3, CL4]

[slides]

Lecture 8

Deep Belief Network

·        
Sampling

·        
Restricted Boltzmann Machine

·        
Deep Belief Network

·        
Applications DBN

Suggested
Readings:

۱٫    [Reducing
the Dimensionality of Data]

۲٫    [A practical
guide to training RBM]

۳٫    [DeeBNet]

۴٫    [FEPCD]

[slides] [video]

Lecture 9

Autoencoders

·        
Sparse Autoencoders

·        
Denoising autoencoders

·        
Contractive Autoencoders

·        
Applications of Autoencoders

Suggested
Readings:

۱٫    [Sparse autoencoder]

۲٫    [Deep learning]

۳٫    [From
PCA to Autoencoders]

[slides] [video]

Lecture 10

Convolutional Neural Networks
۱

·        
Convolutional

·        
Pooling

Suggested
Readings:

۱٫    [ConvNet
notes]

۲٫    [Wikipedia]

[slides] [video]

Lecture 11

Convolutional Neural Networks
۲

·        
AlexNet

·        
ZF Net

·        
VGG Net

·        
GoogLeNet

·        
ResNet

Suggested
Readings:

۱٫    [AlexNet]

۲٫    [ZF Net]

۳٫    [VGG Net]

۴٫    [GoogLeNet]

۵٫    [ResNet]

[slides] [video]

Lecture 12

Convolutional Neural Networks
۳

·        
R-CNN

·        
Fast R-CNN

·        
GAN

Suggested
Readings:

۱٫    [R-CNN]

۲٫    [Fast R-CNN]

۳٫    [GAN]

[slides1] [slides2] 
[video]

Lecture 13

Computational Graphs

·        
Computational Graph Language

·        
Derivative using Computational Graph

·        
Factoring Paths

·        
Forward- and Reverse-Mode Differentiation

Suggested
Readings:

۱٫    [Calculus
on Computational Graphs: Backpropagation]

۲٫    [Computation
Graphs]

[slides] [video]

Lecture 14

Natural Language
Processing

·        
Word Vectors

·        
SVD Based Methods

·        
Iteration Based Methods – Word2vec

Suggested
Readings:

۱٫    [NLP
Notes
۱]

۲٫    [NLP
Notes 2]

[slides] [video]

Lecture 15

Recurrent Neural
Networks

·        
RNN computation

·        
Character-Level Language Models

·        
Backpropagation through time

·        
RNN Examples

Suggested
Readings:

۱٫    [Neural
Nets Lecture 10]

۲٫    [The
Unreasonable Effectiveness of RNNs]

۳٫    [Computational
Graph,
RNN, CNN]

[slides] [video]

Lecture 16

LSTM and GRU

·        
Gradient Vanishing

·        
Long Short Term Memory

·        
Variants on LSTM

Suggested
Readings:

۱٫    [Neural
Nets Lecture 10]

۲٫    [wikipedia]

۳٫    [Understanding
LSTM Networks]

۴٫    [Neural
Networks for Machine Learning]

۵٫    [RNN
Tutorial]

[slides] [video]

Lecture 17

Neural Text Generation

·        
Machine Translation

·        
Bidirectional LSTM

·        
Attention Mechanism

·        
Google’s Multilingual NMT

Suggested
Readings:

۱٫    [Neural
machine translation]

۲٫    [Attention-based
NMT]

۳٫    [Google’s
multilingual NMT]

۴٫    [Google’s NMT ]

[slides] [video]

Lecture 18

Automatic Speech
Recognition (ASR)

·        
Major Speech Processing Tasks

·        
Automatic Speech Recognition

·        
Early DNN based ASR systems

·        
End-to-End DNN based ASR sys.

·        
End-to-End Speech Synthesis

Suggested
Readings:

۱٫    [ASR and DL]

۲٫    [End-to-end
models for Speech]

۳٫    [end-to-end
ASR]

[slides]

Lecture 19

Deep Reinforcement
Learning

·        
Reinforcement Learning

·        
Mathematical formulation of the RL problem

·        
Deep Q-learning

·        
Deep Q-learning examples

Suggested
Readings:

۱٫    [CNN,
Lecture 14]

۲٫    [DRL,
Nature]

۳٫    [DRL,
Vehicle Intelligence Lab]

[slides] [video]

Subject

Note

Assignment #1

Activation Functions

[Assignment #1] [code]

Assignment #2

Simple Neural Network

[Assignment #2]

Suggested Readings:

·        
[Tensorflow
and Deep Learning – without a PhD]

Assignment #3

PGM and DBN

[Assignment #3]

Assignment #4

Word Embedding

[Assignment #4] [Data + Tensorflow]