Deep Learning Book Ch 8 Optimization for Training Deep Models Chapter Link Ch 8 Optimization for Training Deep Models Presentation Slides Online Optimization for Training Deep Models Presentation by Faizan Shaikh SF Optimization for Training Deep Models Presentation by Safak Ozkan Online Discussion Part 1 Part 2 Additional Resources Surrogate Loss Functions in Machine Learning An overview of gradient descent optimization algorithms Layer Normalization Recurrent Highway Networks