Spring 2020 / DM873
Deep Learning

General Information

Machine learning has become a part in our everydays life, from simple product recommendations to personal electronic assistant to self-driving cars. More recently, through the advent of potent hardware and cheap computational power, “Deep Learning” has become a popular and powerful tool for learning from complex, large-scale data.

In this course, we will discuss the fundamentals of deep learning and its application to various different fields. We will learn about the power but also the limitations of these deep neural networks. At the end of the course, the students will have significant familiarity with the subject and will be able to apply the learned techniques to a broad range of different fields.

Mainly, the following topics will be covered:

• feedforward neural networks
• recurrent neural networks
• convolutional neural networks
• backpropagation algorithm
• regularization
• factor analysis

Lectures

Exercises

Project

Aim of the Project

In light of the recent COVID-19 pandemic, we will take a look at one of the severe symptoms of the disease, pneumonia. The dataset we have consists of x-ray images of patients who either have pneumonia or are healthy. You must make your own model to detect whether a patient has developed pneumonia, or is healthy.

To spice things up and to also give you deeper insights in KERAS, you are also supposed to implement your own layers and mix them with existing other layers. Make use of the upcoming Q&A sessions in case you are stuck and require help from Mathias.

You are supposed to work in groups of 4-5 people. In light of the ongoing lock-down, this might be a bit more challenging than usual. Nevertheless, it will help you since you can spread the tasks amongst the team members. In the case you have trouble finding members, please write Mathias to work out a solution.

Further Information

You will find all necessary information in the project description. It is fundamentally important that you start early on, since the models might take some time to train.

Deadline

• Project Hand-In:
  Tuesday the 24th of May 2020 at 23:59.
  This includes:
  • Project report via Blackboard
  • Your code as nextcloud-link via email
  • Your model (as .h5 file) as nextcloud-link via email

Downloads

• Project Description
• Dataset (link to nextcloud)

Procedure of the oral exam

Due to the ongoing Corona crisis, the exam will be conducted online via Zoom

The exam will last about 15-20 minutes. At the beginning, one topic from the list below will be drawn randomly using a random number generator. For each topic the examinee should be prepared to make a short presentation of 5 minutes (sharp). The examinee will have 2 minutes to briefly study the notes for the drawn topic before the presentation. The notes may be consulted during the presentation if needed but it will negatively influence the evaluation of the examinee's performance. The student can either use a presentation or a shared whiteboard for quick sketches, etc. (see technical details below).

After the short presentation, additional questions about the presentation's topic but also about other topics in the curriculum will be asked.

Technical Details and precise procedure:

1. We will have aZoom meeting. Please join well ahead of time (since there might be no-shows). You will be placed in the waiting room.
2. In the waiting room, please monitor the chat to see when it will be your turn.
3. Please log-in with your full name such that we can identify you
4. Once connected, please enable your webcam and microphone. Have your student-ID (or any other photo-ID) ready
5. We will use a random generator and assign your topic
6. While you prepare for your talk (1-2 mins), we will send you a link to a shared whiteboard
7. We have decided to use Jamboard (https://jamboard.google.com/). Give it a try and feel comfortable with it. Also note that there is a virtual laser pointer where you can highlight things in your drawing.
8. While giving your presentation, you can either choose to use this whiteboard or share your screen and use prepared power-point slides
9. Either way, please be prepared and maybe organize a test run with your friends see whether you presentation is suitable for an online meeting
10. After the exam you will receive your grade and put back in the waiting room. You are then free to leave
11. Also, please read through the official guidelines published from SDU and in particular make sure that your computer works, in particular your microphone and webcam.

Below is the list of possible topics and some suggested content. The listed content are only suggestions and are not necessarily complete nor must everything be covered in the short presentation. It is the responsibility of the examinee to gather and select among all relevant information for each topic from the course material. On the course website you can find suggested readings for each of these topics.

Topics for the Oral Exam:

1. Feed-Forward Networks
• Function Principle
• Output Units
• Hidden Units
• ...
2. Backpropagation
• Function Principle
• Computational Graphs
• Backpropagation through time
• ...
3. Regularization
• Over/Underfitting & Model Capacity
• Parameter Penalties
• Bagging
• Dropout
• ...
4. Convolutional Neural Networks
• Function Principle
• Pooling
• Initialization of the kernels
• ...
5. Recurrent Neural Networks
• Function Principle
• Problems with long term memory
• Long Short Term Memory
• ...
6. Autoencoders and GANs
• Autoencoders
• Variational Autoencoders
• GANs
• ...
7. Normalization & Style Transfer
• Batch Normalization
• Instance Normalization
• Style Trannsfer Methods
• ...

Materials

• All lecture slides are relevant for the exams.
• All readings noted in the lecture list are relevant for the exam.
• Ian Goodfellow, Yoshua Bengio, Aaron Courville - The Deep Learning Book
• Gareth James, Daniela Witten, Trevor Hastie - Robert Tibshirani An Introduction to Statistical Learning (ISL)