Spring 2019 / DM873
Deep Learning
General Information
Machine learning has become a part in our everydays life, from simple product recommendations to personal electronic assistant to self-driving cars. More recently, through the advent of potent hardware and cheap computational power, “Deep Learning” has become a popular and powerful tool for learning from complex, large-scale data.
In this course, we will discuss the fundamentals of deep learning and its application to various different fields. We will learn about the power but also the limitations of these deep neural networks. At the end of the course, the students will have significant familiarity with the subject and will be able to apply the learned techniques to a broad range of different fields.
Mainly, the following topics will be covered:
- feedforward neural networks
- recurrent neural networks
- convolutional neural networks
- backpropagation algorithm
- regularization
- factor analysis
Project
Aim of the Project
In this assignment you will be taking a look at images of butterflies and their specific butterfly families. Butterflies from different families look different, but even butterflies from the same family can look very different. This is a case of having a high intra-class variance. You will be creating networks to differentiate between these butterflies and classify the specific family of a butterfly, but it is not an easy task.
Further Information
You will find all necessary informaion in the project description. It is fundamentally important that you attend the exercises to discuss your progrss and receive feedback on your timly hand-ins.
Deadlines
- Task 1:
27th of April
- Task 2:
17th of May
- Task 3 & the final deadline:
Tuesday the 4th of June 2019 at 23:59 for all 3 tasks.
Downloads
Procedure of the oral exam
The exam will last about 15-20 minutes. At the beginning, one topic from the list below will be drawn randomly. For each topic the examinee should be prepared to make a short presentation of 5 minutes. It is allowed to bring one page of hand-written notes (DIN A4 or US-Letter, one-sided) for each of the topics. The examinee will have 2 minutes to briefly study the notes for the drawn topic before the presentation. The notes may be consulted during the presentation if needed but it will negatively influence the evaluation of the examinee's performance. During the presentation, only the blackboard can be used (you cannot use overhead transparencies, for instance).
After the short presentation, additional question about the presentation's topic but also about other topics in the curriculum will be asked.
Below is the list of possible topics and some suggested content. The listed content are only suggestions and is not necessarily complete nor must everything be covered in the short presentation. It is the responsibility of the examinee to gather and select among all relevant information for each topic from the course material. On the course website you can find suggested readings for each of these topics.
Topics for the Oral Exam:
- Feed-Forward Networks
- Function Principle
- Output Units
- Hidden Units
- ...
- Backpropagation
- Function Principle
- Computational Graphs
- Backpropagation through time
- ...
- Regularization
- Over/Underfitting & Model Capacity
- Parameter Penalties
- Bagging
- Dropout
- ...
- Convolutional Neural Networks
- Function Principle
- Pooling
- Initialization of the kernels
- ...
- Recurrent Neural Networks
- Function Principle
- Problems with long term memory
- Long Short Term Memory
- ...
- Optimization for Neural Networks
- Parameter Initialization
- Adaptive Learning
- Batch Normalization
- Pre-training
- ...
- Autoencoders and GANs
- Autoencoders
- Variational Autoencoders
- GANs
- ...
