Summer School 2020
Deep Learning

General Information

The Course is held Online. You should already have received information regarding the process. If not, please contact me.

Machine learning has become a part in our everydays life, from simple product recommendations to personal electronic assistant to self-driving cars. More recently, through the advent of potent hardware and cheap computational power, “Deep Learning” has become a popular and powerful tool for learning from complex, large-scale data.

In this course, we will discuss the fundamentals of deep learning and its application to various different fields. We will learn about the power but also the limitations of these deep neural networks. At the end of the course, the students will have significant familiarity with the subject and will be able to apply the learned techniques to a broad range of different fields.

Mainly, the course will cover practical as well theoretical aspects of the Deep Learning and also gives a broad general introduction to machine learning and statistical learning in general.

Materials

The students should bring:

You only need a laptop with a 64 Bit CPU and Python 3+ running
We will form teams and work on the practical parts of the course together, so even if your laptop is very old you can easily participate within a group

Reading Material (please note that the books cover the stuff in more detail than needed for the course):

All lecture slides will be uploaded before the lecture.
The Deep Learning Book (Part I & Part II) (short: DLB)
An Introduction to Statistical Learning (Chapters 1-3) (short: ITSL)

Lectures

Course Overview

Week 1	Monday	Tuesday	Wednesday	Thursday	Friday
Morning	Welcome & Introduction	FF-Networks Linear Algebra	Discussion Ex1 ML Basics	Discussion Ex2 Regularization	Discussion Ex3 Back Prop.
Afternoon	System Setup Python Packages Visualization Tools	Statistics KERAS part 1 Exercise	Linear Reg. Exercise	Ouput Layer Loss Functs Logistic Reg. Exercise	Grad. Descent KERAS part 2

Week 2	Monday	Tuesday	Wednesday	Thursday	Friday
Morning	Recap CNNs	Discussion Ex4 RNNs	Discussion Ex5 Optimzation Strategies	Recap & Q&A Final Project	Final Project
Afternoon	Network Vis. Exercise	Exercise	Outlook Final Project	Final Project	Discussion Final Project Goodbye

Lecture Material

#	Downloads	Comments
Day 1	Introduction
Day 2	Simple Networks - Part 1 Intermezzo: Linear Algebra Simple Networks - Part 2	DLB, Chap. 2,3 & 6 (except 2.8, 2.9, 3.13, 3.14)
Day 3	Statistics Machine Learning Basics Lin. Reg.	ITSL, Chap. 2 & 3
Day 4	Regularization Log. Res.	ITSL, Chap. 4.1 - 4.3, DLB, Chap. 7
Day 5	Gradient Learning Backpropagation	DLB, Chap 6.3 & 6.5
Day 6	Convolution	DLB, Chap. 9
Day 7	RNNs	DLB, Chap. 10
Day 8	Some Optimizations Cool Stuff
Day 9
Day 10

Exercises

You also might like the KERAS Cheat Sheet: here

#	Content	Downloads	Reading	Solution
1	Introduction to python	Questions Dataset		Solution
2	Introduction to KERAS	Questions	Keras Intro	Solution
3	Linear Regression	Questions Dataset		Solution
4	Regularization	Questions		Solution (thanks to Sergej!)
5	More Keras	No Exercises; it's Friady!	Keras Intro
6	Convolutional Networks	Questions	Layer Visualization	Solution (thank to Jona and team) ipynb or html
7	Recurrent Networks	Questions

Final Project

The project task is to tell images of dogs and cats apart.

Please carefully read the instructions here

The Key-Objectives are:

Create an appropriate network
Read the images through a data-generator
Train your model and estimate the generalization error
Think of an appropriate visualization for demonstrating the quality of your model
Comment your code and explain every decision you make in the code
Submit your code and the visualization to Mathias (email in the project description)

Deadline

Hand-in your code before Friday noon (12:00)

Competition

We also host a small competition. The team of the best performing model will receive a ~~Raspberry Pie~~ ... a round of applause ... sorry, Corona-times. Please be fair: Only use the images provided to, do not use any outside images. Further, do not use pre-trained models, your models need to be created and trained from scratch by you. Make sure your submitted model plugs seamlessly into the folowing test-script: here. Further, it does not matter whether you encode cat or dog as 1: In case of a different encoding, we can flip the accuracy value.

Dataset

Please use the following dataset and ONLY this dataset: dataset

Summer School 2020Deep Learning