Intelligent and Interactive Systems

User Tools

Site Tools



Advanced Machine Learning (VO2 + PS1, Master)

See the UIBK course catalog entries for the lecture and the seminar for essential information. See also the companion course Probabilistic Models and Inference (VO1 + PS1, Master).

This course (VO+PS) can be allocated for FLAT (VO+PS) or Compiler Construction (VO+PS). If and only if this is what you intend, then register for Modern Aspects in Computer Science: Data Science (VO+PS) instead.

The course content depends on the audience:

  • If the majority of participants has taken Machine Learning (VO3 + PS2, Bachelor) or equivalent, then this course will cover the basics rapidly as needed, and then proceed with more sophisticated machine learning methods.
  • If the majority of participants has only little or no prior knowledge, then this course will cover the basics thoroughly, and course content will overlap significantly with Machine Learning (VO3 + PS2, Bachelor).
  • Besides the core content, students will be able to vote for some topics to be taught.

Course Concept

The course takes place online. Find more information in OLAT on the VO and the PS.

The first one or two sessions will take place like a regular lecture, just online. Then, to maximize the benefits of online teaching we will proceed as follows:

  • Prior to all following sessions I will record and upload a sequence of brief videos totaling up to 60 minutes.
  • Students will be required to study the material prior to each class session, by watching the prerecorded videos, by reading the given material, or a combination of both.
  • The lecture part of the class sessions will be shortened to 30 minutes, and will be dedicated to highlighting key issues and answering questions.
  • The PS part of the class sessions will be dedicated to discussing exercises, which are to be completed offline.

Students are strongly encouraged to form study groups who work together (possibly online according to the regulations) for self-study and on ungraded exercises.

There is an online forum for students to ask and answer each other's questions, where also the instructors will answer questions.


The course will be based on the following texts, all freely available online:

A very accessible textbook on many of the topics covered in this and the companion course is Machine Learning: A Probabilistic Perspective by Kevin Murphy, MIT Press 2012.


The topics to be covered are negotiable. If you would like specific topics covered, please contact the instructor.

Videos are in OLAT.

Any entries pertaining to the future should be considered indicative and may change any time without notice.

Date Mandatory Preparation Topic Lecture Notes
2020-10-27 RL 1.1-1.6, 3.1-3.5; Videos 200, 202 (video names updated, content is unchanged) Reinforcement Learning: Introduction, Markov Decision Process RL Notes; RL Day 1 Slides
2020-11-03 RL 3.6-3.8, 4.1-4.4; Videos 210, 212, 214, 216, 218 Reinforcement Learning: Bellman Equations, Dynamic Programming RL Notes; RL Day 2 Slides; RL Day 2 Workbook
2020-11-10 RL 5.1, 6.1,6.2,6.4,6.5, 9.1,9.2,9.3,9.4,9.5, 10.1, 11.1,11.3; Videos 220, 222, 224, 226, 228, 229 Reinforcement Learning: Model-free Prediction and Control, Value Function Approximation RL Notes; RL Day 3 Slides
2020-11-17 RL 13.1-13.5,13.7,13.8; Videos 230, 232, 234, 236 Reinforcement Learning: Policy Gradients and Actor Critic methods RL Notes; RL Day 4 Slides

Topic Video Text Notes Ex.
2020-10-06Linear Regression
Introduction intro
Linear Regression PRML 1.1 PRML-01
2020-10-13Linear Regression: Regularization, Solving; Cross Validation
Regression: Overfitting and Regularization 2 35:09 PRML 1.1 PRML-01 +
Regression: Solving for a first-order polynomial 4 14:22 PRML 1.1 PRML-01 +
Model Selection: Cross-Validation 6 9:13 PRML 1.3 PRML-01 +
2020-10-20Linear Regression: Basis Functions and Design Matrix; K-Means Clustering
Regression: Basis Functions 16 17:33 PRML 3.1 (Intro) PRML-01 +
Regression: Solving with Design Matrix 18 10:14 PRML 3.1.1 PRML-01 +
Regression: Solving Regularized Regression with Design Matrix 20 8:12 PRML 3.1.4 PRML-01 +
Clustering: K-Means 22 22:48 PRML 9.1 kmeans +
2020-11-24Neural Networks: Intro; Forward Propagation; Squared Error Function
Neural Networks: Preface 300 8:42 nn
Neural Networks: Model Neuron and Activation Functions 302 5:44 nn
Neural Networks: Terminology and Notation 304 8:51 nn
Neural Networks: Forward Propagation 306 9:19 nn +
Neural Networks: Remarks 308 7:37 nn
Neural Networks: Universal Approximation Property 310 6:18 PRML Fig. 5.3 nn
Gradient Descent 312 6:45 gradient-descent +
Neural Networks: Squared Error Function 314 08:01 nn
2020-12-01Neural Networks: Gradient-Descent Training by Error Backpropagation
Chain Rule for Simple and Partial Derivatives 316 5:29 nn +
Neural Networks: Error Backpropagation via the Chain Rule 318 8:37 nn +
Neural Networks: Computing the Deltas Layer by Layer 320 6:38 nn +
Neural Networks: The Batch Gradient Descent Training Algorithm 322 5:39 nn +
Neural Networks: Variants of Gradient-Descent Training Methods 324 9:38 nn
2020-12-15Neural Netwoks: Cost Functions; Implementation Tricks; Convolution
Neural Networks: Cost Functions and Output Units 326 17:45 PRML 5.2 (Intro); DL 6.2 nn Sect. 4 +
Neural Networks: Getting Training to Perform Well 328 14:39 DL 8.7 nn Sect. 5
Neural Networks: Smoothing (to Motivate Convolution) 330 10:17 nn Sect. 6
Neural Networks: Convolution 332 12:53 nn Sect. 6
Convolutional Neural Networks: Introduction 334 12:10 DL 9.2 nn Sect. 7
Convolutional Neural Networks: Forward Propagation 336 17:08 nn Sect. 7.7
Convolutional Neural Networks: Demonstration 338 7:17 nn Sect. 7.7, 7.8 +
2021-01-12Convolutional and Autoencoder Networks
Convolutional Neural Networks: Pooling; Typical Architecture 340 6:05 nn Sect. 7.97.12
Convolutional Neural Networks: Example Architecture and Features 342 9:05 nn Sect. 7.13, 7.14
Convolutional Neural Networks: Error Backpropagation 344 9:24 nn Sect. 7.15, 7.16 +
Convolutional Neural Networks: Variants of Connectivity 346 6:07 nn Sect. 8
Neural Networks: Sparse Autoencoder; Kullback-Leibler Divergence 350 12:18 nn Sect. 9 +
Neural Networks: Features learned by a sparse autoencoder 352 6:08 nn Sect. 9.5 +
2021-01-19Kernel Methods
Kernel Methods PRML 6–6.2 PRML-06 Sect. 2, 3, 5.1 +
Lagrange Multipliers PRML Appendix E Lagrange-multipliers +
2021-01-26Gaussian Process Regression
Gaussian Process Regression PRML 6.4–6.4.3, 6.4.5 PRML-06 Sect. 4 +
2021-02-02Support Vector Machines
Support Vector Machines PRML 7–7.1.1, 7.1.3 PRML-07 +

RSS Feed

Justus Piater, 2020/10/11 22:28

The preparation material for 2020-10-13 has been finalized.

Justus Piater, 2020/10/15 21:02

The preparation material for 2020-10-20 has been finalized.

Sayantan Auddy, 2020/10/22 23:34

The preparation material for 2020-10-27 has been finalized.

Sayantan Auddy, 2020/10/28 12:59

The preparation material for 2020-11-03 has been finalized.

Justus Piater, 2020/10/28 14:22

This page is now accessible from the internet; a VPN is no longer required.

Sayantan Auddy, 2020/11/06 17:57

The preparation material for 2020-11-10 has been finalized.

Sayantan Auddy, 2020/11/14 02:35

The preparation material for 2020-11-17 has been finalized.

Justus Piater, 2020/11/21 22:03

The preparation material for 2020-11-24 has been finalized.

Justus Piater, 2020/11/29 09:51, 2020/11/29 09:53

The preparation material for 2020-12-01 has been finalized. Please do the exercises before class!

Justus Piater, 2020/12/11 17:45

The preparation material for 2020-12-15 has been finalized. Please always do the exercises before class!

Justus Piater, 2021/01/10 20:23

The preparation material for 2021-01-12 has been finalized.

Justus Piater, 2021/01/16 13:15

The preparation material for 2021-01-19 has been finalized.

Please read the given sections in the book, try to follow the math, and give the exercises referred to in the course notes a shot. We will look at these on Tuesday.

Justus Piater, 2021/01/19 18:00

Oral exams will take place February 10 and 11 online. Please register for one of the two dates by January 27.

Online exams are subject to the conditions Absolvierung der kommissionellen Abschlussprüfung bzw. Defensio. In addition:

  • You need to be able to write and draw in a way that I can see it on my screen. Thus, you need either a tablet (possibly separately connected to the video call), a webcam facing a sheet of paper or a board, or equivalent. Please arrange and test this beforehand!
  • I will occasionally take screenshots of the video call for my records (no video, no voice).
Justus Piater, 2021/01/25 11:11

The preparation material for the rest of the semester has been finalized. Please read the given sections in the book, try to follow the math, and give the exercises referred to in the course notes a shot. We will look at these again in class.

Justus Piater, 2021/01/28 12:01

The oral exam schedule can be found in OLAT.

Justus Piater, 2021/03/23 12:02

The next round of oral exams will take place May 4th. Please register by April 20. They will take place online, under the same conditions as the first round.

Justus Piater, 2021/04/21 20:03

The oral exam schedule can be found in OLAT.

Enter your comment. Wiki syntax is allowed:
courses/2020w/703642/start.txt · Last modified: 2021/02/07 17:16 by Justus Piater