See the UIBK course catalog entries for the lecture and the seminar for essential information. See also the companion course Probabilistic Models and Inference (VO1 + PS1, Master).
This course (VO+PS) can be allocated for FLAT (VO+PS) or Compiler Construction (VO+PS). If and only if this is what you intend, then register for Modern Aspects in Computer Science: Data Science (VO+PS) instead.
The course content depends on the audience:
The first one or two sessions will take place like a regular lecture, just online. Then, to maximize the benefits of online teaching we will proceed as follows:
Students are strongly encouraged to form study groups who work together (possibly online according to the regulations) for self-study and on ungraded exercises.
There is an online forum for students to ask and answer each other's questions, where also the instructors will answer questions.
The course will be based on the following texts, all freely available online:
A very accessible textbook on many of the topics covered in this and the companion course is Machine Learning: A Probabilistic Perspective by Kevin Murphy, MIT Press 2012.
The topics to be covered are negotiable. If you would like specific topics covered, please contact the instructor.
Videos are in OLAT.
Where “HTML; PDF” are available, the HTML slides are exactly what you see in class; you should download the PDF notes because they contain additional information. The exercises included in the notes are useful to consolidate the material presented in class; many of them are done in class.
In addition to the resources given below, this course extensively uses Bishop's slides for PRML chapter 1.
Any entries pertaining to the future should be considered indicative and may change any time without notice.
|Date||Mandatory Preparation||Topic||Lecture Notes|
|2020-10-27||RL 1.1-1.6, 3.1-3.5; Videos 200, 202 (video names updated, content is unchanged)||Reinforcement Learning: Introduction, Markov Decision Process||RL Notes; RL Day 1 Slides|
|2020-11-03||RL 3.6-3.8, 4.1-4.4; Videos 210, 212, 214, 216, 218||Reinforcement Learning: Bellman Equations, Dynamic Programming||RL Notes; RL Day 2 Slides; RL Day 2 Workbook|
|2020-11-10||RL 5.1, 6.1,6.2,6.4,6.5, 9.1,9.2,9.3,9.4,9.5, 10.1, 11.1,11.3; Videos 220, 222, 224, 226, 228, 229||Reinforcement Learning: Model-free Prediction and Control, Value Function Approximation||RL Notes; RL Day 3 Slides|
|2020-11-17||RL 13.1-13.5,13.7,13.8; Videos 230, 232, 234, 236||Reinforcement Learning: Policy Gradients and Actor Critic methods||RL Notes; RL Day 4 Slides|
|Linear Regression||PRML 1.1||PRML-01|
|2020-10-13Linear Regression: Regularization, Solving; Cross Validation|
|Regression: Overfitting and Regularization||2 35:09||PRML 1.1||PRML-01||+|
|Regression: Solving for a first-order polynomial||4 14:22||PRML 1.1||PRML-01||+|
|Model Selection: Cross-Validation||6 9:13||PRML 1.3||PRML-01||+|
|2020-10-20Linear Regression: Basis Functions and Design Matrix; K-Means Clustering|
|Regression: Basis Functions||16 17:33||PRML 3.1 (Intro)||PRML-01||+|
|Regression: Solving with Design Matrix||18 10:14||PRML 3.1.1||PRML-01||+|
|Regression: Solving Regularized Regression with Design Matrix||20 8:12||PRML 3.1.4||PRML-01||+|
|Clustering: K-Means||22 22:48||PRML 9.1||kmeans||+|
|2020-11-24Neural Networks: Intro; Forward Propagation; Squared Error Function|
|Neural Networks: Preface||300 8:42||nn|
|Neural Networks: Model Neuron and Activation Functions||302 5:44||nn|
|Neural Networks: Terminology and Notation||304 8:51||nn|
|Neural Networks: Forward Propagation||306 9:19||nn||+|
|Neural Networks: Remarks||308 7:37||nn|
|Neural Networks: Universal Approximation Property||310 6:18||PRML Fig. 5.3||nn|
|Gradient Descent||312 6:45||gradient-descent||+|
|Neural Networks: Squared Error Function||314 08:01||nn|
|2020-12-01Neural Networks: Gradient-Descent Training by Error Backpropagation|
|Chain Rule for Simple and Partial Derivatives||316 5:29||nn||+|
|Neural Networks: Error Backpropagation via the Chain Rule||318 8:37||nn||+|
|Neural Networks: Computing the Deltas Layer by Layer||320 6:38||nn||+|
|Neural Networks: The Batch Gradient Descent Training Algorithm||322 5:39||nn||+|
|Neural Networks: Variants of Gradient-Descent Training Methods||324 9:38||nn|
|2020-12-15Neural Netwoks: Cost Functions; Implementation Tricks; Convolution|
|Neural Networks: Cost Functions and Output Units||326 17:45||PRML 5.2 (Intro); DL 6.2||nn Sect. 4||+|
|Neural Networks: Getting Training to Perform Well||328 14:39||DL 8.7||nn Sect. 5|
|Neural Networks: Smoothing (to Motivate Convolution)||330 10:17||nn Sect. 6|
|Neural Networks: Convolution||332 12:53||nn Sect. 6|
|Convolutional Neural Networks: Introduction||334 12:10||DL 9.2||nn Sect. 7|
|Convolutional Neural Networks: Forward Propagation||336 17:08||nn Sect. 7.7|
|Convolutional Neural Networks: Demonstration||338 7:17||nn Sect. 7.7, 7.8||+|
|2021-01-12Convolutional and Autoencoder Networks|
|Convolutional Neural Networks: Pooling; Typical Architecture||340 6:05||nn Sect. 7.9–7.12|
|Convolutional Neural Networks: Example Architecture and Features||342 9:05||nn Sect. 7.13, 7.14|
|Convolutional Neural Networks: Error Backpropagation||344 9:24||nn Sect. 7.15, 7.16||+|
|Convolutional Neural Networks: Variants of Connectivity||346 6:07||nn Sect. 8|
|Neural Networks: Sparse Autoencoder; Kullback-Leibler Divergence||350 12:18||nn Sect. 9||+|
|Neural Networks: Features learned by a sparse autoencoder||352 6:08||nn Sect. 9.5||+|
|Kernel Methods||PRML 6–6.2||PRML-06 Sect. 2, 3, 5.1||+|