Summer School 2020
Deep Learning

General Information

The Course is held Online. You should already have received information regarding the process. If not, please contact me.

Machine learning has become a part in our everydays life, from simple product recommendations to personal electronic assistant to self-driving cars. More recently, through the advent of potent hardware and cheap computational power, “Deep Learning” has become a popular and powerful tool for learning from complex, large-scale data.

In this course, we will discuss the fundamentals of deep learning and its application to various different fields. We will learn about the power but also the limitations of these deep neural networks. At the end of the course, the students will have significant familiarity with the subject and will be able to apply the learned techniques to a broad range of different fields.

Mainly, the course will cover practical as well theoretical aspects of the Deep Learning and also gives a broad general introduction to machine learning and statistical learning in general.

Materials

The students should bring:
  • You only need a laptop with a 64 Bit CPU and Python 3+ running
  • We will form teams and work on the practical parts of the course together, so even if your laptop is very old you can easily participate within a group
Reading Material (please note that the books cover the stuff in more detail than needed for the course):

Lectures

Course Overview

Week 1 Monday Tuesday Wednesday Thursday Friday
Morning Welcome &
Introduction
FF-Networks
Linear Algebra
Discussion Ex1
ML Basics
Discussion Ex2
Regularization
Discussion Ex3
Back Prop.
Afternoon System Setup
Python Packages
Visualization Tools
Statistics
KERAS part 1
Exercise
Linear Reg.
Exercise
Ouput Layer
Loss Functs
Logistic Reg.
Exercise
Grad. Descent
KERAS part 2
Week 2 Monday Tuesday Wednesday Thursday Friday
Morning Recap
CNNs
Discussion Ex4
RNNs
Discussion Ex5
Optimzation Strategies
Recap & Q&A
Final Project
Final Project
Afternoon Network Vis.
Exercise
Exercise Outlook
Final Project
Final Project Discussion Final Project
Goodbye

Lecture Material

# Downloads Comments
Day 1 Introduction
Day 2 Simple Networks - Part 1
Intermezzo: Linear Algebra
Simple Networks - Part 2
DLB, Chap. 2,3 & 6
(except 2.8, 2.9, 3.13, 3.14)
Day 3 Statistics
Machine Learning Basics
Lin. Reg.
ITSL, Chap. 2 & 3
Day 4 Regularization
Log. Res.
ITSL, Chap. 4.1 - 4.3, DLB, Chap. 7
Day 5 Gradient Learning
Backpropagation
DLB, Chap 6.3 & 6.5
Day 6 Convolution DLB, Chap. 9
Day 7 RNNs DLB, Chap. 10
Day 8 Some Optimizations
Cool Stuff
Day 9
Day 10

Final Project

The project task is to tell images of dogs and cats apart.

Please carefully read the instructions here

The Key-Objectives are:

  • Create an appropriate network
  • Read the images through a data-generator
  • Train your model and estimate the generalization error
  • Think of an appropriate visualization for demonstrating the quality of your model
  • Comment your code and explain every decision you make in the code
  • Submit your code and the visualization to Mathias (email in the project description)

Deadline

Hand-in your code before Friday noon (12:00)

Competition

We also host a small competition. The team of the best performing model will receive a Raspberry Pie ... a round of applause ... sorry, Corona-times. Please be fair: Only use the images provided to, do not use any outside images. Further, do not use pre-trained models, your models need to be created and trained from scratch by you. Make sure your submitted model plugs seamlessly into the folowing test-script: here. Further, it does not matter whether you encode cat or dog as 1: In case of a different encoding, we can flip the accuracy value.

Dataset

Please use the following dataset and ONLY this dataset: dataset