Advanced Machine Learning (VO2 + PS1, Master)

See the UIBK course catalog entries for the lecture and the seminar for essential information. See also the companion course Probabilistic Models and Inference (VO1 + PS1, Master).

This course (VO+PS) can be allocated for FLAT (VO+PS) or Compiler Construction (VO+PS). If and only if this is what you intend, then register for Modern Aspects in Computer Science: Data Science (VO+PS) instead.

The course content depends on the audience:

If the majority of participants has taken Machine Learning (VO3 + PS2, Bachelor) or equivalent, then this course will cover the basics rapidly as needed, and then proceed with more sophisticated machine learning methods.
If the majority of participants has only little or no prior knowledge, then this course will cover the basics thoroughly, and course content will overlap significantly with Machine Learning (VO3 + PS2, Bachelor).
Besides the core content, students will be able to vote for some topics to be taught.

Course Concept

The course takes place online. Find more information in OLAT on the VO and the PS.

The first one or two sessions will take place like a regular lecture, just online. Then, to maximize the benefits of online teaching we will proceed as follows:

Prior to all following sessions I will record and upload a sequence of brief videos totaling up to 60 minutes.
Students will be required to study the material prior to each class session, by watching the prerecorded videos, by reading the given material, or a combination of both.
The lecture part of the class sessions will be shortened to 30 minutes, and will be dedicated to highlighting key issues and answering questions.
The PS part of the class sessions will be dedicated to discussing exercises, which are to be completed offline.

Students are strongly encouraged to form study groups who work together (possibly online according to the regulations) for self-study and on ungraded exercises.

There is an online forum for students to ask and answer each other's questions, where also the instructors will answer questions.

Textbooks

The course will be based on the following texts, all freely available online:

PRML Pattern Recognition and Machine Learning by Chris Bishop, Springer 2006
RL Reinforcement Learning: An Introduction by Rich Sutton and Andy Barto, MIT Press 2018
DL Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville, MIT Press 2016 (PDF)

A very accessible textbook on many of the topics covered in this and the companion course is Machine Learning: A Probabilistic Perspective by Kevin Murphy, MIT Press 2012.

Agenda

The topics to be covered are negotiable. If you would like specific topics covered, please contact the instructor.

Videos are in OLAT.

Where “HTML; PDF” are available, the HTML slides are exactly what you see in class; you should download the PDF notes because they contain additional information. The exercises included in the notes are useful to consolidate the material presented in class; many of them are done in class.

In addition to the resources given below, this course extensively uses Bishop's slides for PRML chapter 1.

Any entries pertaining to the future should be considered indicative and may change any time without notice.

Date	Mandatory Preparation	Topic	Lecture Notes
2020-10-27	RL 1.1-1.6, 3.1-3.5; Videos 200, 202 (video names updated, content is unchanged)	Reinforcement Learning: Introduction, Markov Decision Process	RL Notes; RL Day 1 Slides
2020-11-03	RL 3.6-3.8, 4.1-4.4; Videos 210, 212, 214, 216, 218	Reinforcement Learning: Bellman Equations, Dynamic Programming	RL Notes; RL Day 2 Slides; RL Day 2 Workbook
2020-11-10	RL 5.1, 6.1,6.2,6.4,6.5, 9.1,9.2,9.3,9.4,9.5, 10.1, 11.1,11.3; Videos 220, 222, 224, 226, 228, 229	Reinforcement Learning: Model-free Prediction and Control, Value Function Approximation	RL Notes; RL Day 3 Slides
2020-11-17	RL 13.1-13.5,13.7,13.8; Videos 230, 232, 234, 236	Reinforcement Learning: Policy Gradients and Actor Critic methods	RL Notes; RL Day 4 Slides

Topic	Video	Text	Notes	Ex.
2020-10-06Linear Regression
Introduction			intro
Linear Regression		PRML 1.1	PRML-01
2020-10-13Linear Regression: Regularization, Solving; Cross Validation
Regression: Overfitting and Regularization	2 35:09	PRML 1.1	PRML-01	+
Exercises: What is the distinction between classification and regression? Why do we do regression? What is the (application-level) problem it solves? What do we ultimately want to achieve? Why does our error function of choice minimize the sum of squared differences as opposed to, say, the sum of absolute differences? What is underfitting and how do we recognize it? What is overfitting and how do we recognize it? Why can't we directly find the best value of \(M\) or of \(\lambda\) by choosing it to minimize the red curve on Bishop slides 9 or 16, respectively? Why does overfitting tend to come with large magnitudes of coefficients? Why is it better to adjust \(\lambda\) than to balance \(M\) and the number of training instances? The polynomial coefficient \(w_0\) is special. In what way? What implications does this have for regularization?
Regression: Solving for a first-order polynomial	4 14:22	PRML 1.1	PRML-01	+
Exercises: Work out the result for a zeroth-order polynomial. How does our result generalize to higher-order polynomials? What do the coefficient matrix, the vector of unknowns and the vector of constants look like in general? What methods would you use to solve such a system in general? Draw on your math knowledge and external resources as needed.
Model Selection: Cross-Validation	6 9:13	PRML 1.3	PRML-01	+
Exercises: Why shouldn't we evaluate models simply in terms of training-/test-/validation-set performance? What is the main drawback of cross-validation? Why does model selection involve more than just choosing the best-performing model?
2020-10-20Linear Regression: Basis Functions and Design Matrix; K-Means Clustering
Regression: Basis Functions	16 17:33	PRML 3.1 (Intro)	PRML-01	+
Exercises: How do we express regression with a quadratic polynomial with basis functions? How do we express regression with a quadratic polynomial over a 2D domain with basis functions? Expressing our regression problem as \(A \mathbf{w} = b\), verify that \(\Phi^{\textrm{T}} \Phi = A\), and that \(\Phi^{\textrm{T}} t = b\). What is a key drawback of fitting polynomials in many applications? How can we overcome this using basis functions?
Regression: Solving with Design Matrix	18 10:14	PRML 3.1.1	PRML-01	+
Exercises: What is the distinction between \(y(x; \mathbf{w})\) and \(\mathbf{y}\)? Make sure you understand all the (matrix-vector) arithmetic and how the derivative is calculated.
Regression: Solving Regularized Regression with Design Matrix	20 8:12	PRML 3.1.4	PRML-01	+
Exercises: Make sure you understand all the (matrix-vector) arithmetic. Why is this regularization scheme called ridge regression?
Clustering: K-Means	22 22:48	PRML 9.1	kmeans	+
Exercises: Prove that the cluster centers that minimize the reconstruction error are the arithmetic cluster means. Why does \(K\)-Means converge? What does it converge to? In the example run of \(K\)-Means, the axis scaling is different than in the original Old-Faithful data. Why? What happens if the data exhibit two clearly-distinct clusters of very different spatial extent?
2020-11-24Neural Networks: Intro; Forward Propagation; Squared Error Function
Neural Networks: Preface	300 8:42		nn
Neural Networks: Model Neuron and Activation Functions	302 5:44		nn
Neural Networks: Terminology and Notation	304 8:51		nn
Neural Networks: Forward Propagation	306 9:19		nn	+
Exercises: Write down the fully-expanded forward pass (\(\mathbf{h}_{W,\mathbf{b}}(\mathbf{x}) = \cdots\) as a function of the weight matrices and bias vectors) for the example network with two hidden layers and two output units. What are the dimensions of the weight matrices and bias vectors? Show that any feed-forward neural network with identity activation functions \(f(z) = z\) can be replaced by an equivalent neural network without hidden layers. What are the weight matrix and the bias vector of this network? Argue that this generalizes to any linear activation function \(f(z)\).
Neural Networks: Remarks	308 7:37		nn
Neural Networks: Universal Approximation Property	310 6:18	PRML Fig. 5.3	nn
Gradient Descent	312 6:45		gradient-descent	+
Exercises: For functions on two- and higher-dimensional domains, would you expect the problem of local minima to be more or less severe than for functions on one-dimensional domains? For functions on two- and higher-dimensional domains, under what circumstances can the direction of fastest progress differ from the local gradient? In fact, under such circumstances gradient descent may progress arbitrarily slowly even though there is a direction of much faster progress.
Neural Networks: Squared Error Function	314 08:01		nn
2020-12-01Neural Networks: Gradient-Descent Training by Error Backpropagation
Chain Rule for Simple and Partial Derivatives	316 5:29		nn	+
Exercises: Consider a function \(f = f(x, y)\) with \(x = x(u, v)\), \(y = y(u, v)\), \(u = u(t)\), \(v = v(t)\). Draw the graphical representation of this nested function, and work out \(\frac{\mathrm{d} f}{\mathrm{d} t}\).
Neural Networks: Error Backpropagation via the Chain Rule	318 8:37		nn	+
Exercises: Why is \(E\left(w_{i j}^{(l)}\right) = E\left(z_i^{(l+1)}\left(w_{i j}^{(l)}\right)\right)\)? Instead of going through \(z_i^{(l+1)}\), could we have chosen to expand \(E\left(w_{i j}^{(l)}\right)\) through some other variable?
Neural Networks: Computing the Deltas Layer by Layer	320 6:38		nn	+
Exercises: Fill in the details in the calculation of \(\delta_i^{(L)}\). Fill in the details in the calculation of \(\delta_j^{(l)}\).
Neural Networks: The Batch Gradient Descent Training Algorithm	322 5:39		nn	+
Exercises: Fill in the details in the calculation of \(\frac{\partial}{\partial w_{i j}^{(l)}} E(W,\mathbf{b}; \mathbf{x}, \mathbf{t})\) and \(\frac{\partial}{\partial b_i^{(l)}} E(W,\mathbf{b}; \mathbf{x}, \mathbf{t})\). Do the interactive exercise. Show mathematically that the parameter update of the Batch Gradient Descent algorithm in fact descends the gradient of \(E(W, \mathbf{b})\) as defined on the Error Function slide.
Neural Networks: Variants of Gradient-Descent Training Methods	324 9:38		nn
2020-12-15Neural Netwoks: Cost Functions; Implementation Tricks; Convolution
Neural Networks: Cost Functions and Output Units	326 17:45	PRML 5.2 (Intro); DL 6.2	nn Sect. 4	+
Exercises: For the backpropagation algorithm, work out \(\delta^{(L)}\) for the Normal output distribution and the identity activation function at the output layer. For the backpropagation algorithm, work out \(\delta^{(L)}\) for the Bernoulli output distribution and the logistic sigmoid activation function at the output layer. Begin by showing that \(\frac{\mathrm{d}}{\mathrm{d}z}\sigma(z) = \sigma(z) (1 - \sigma(z))\). For the backpropagation algorithm, work out \(\delta^{(L)}\) for the Multinoulli output distribution and a softmax output layer.
Neural Networks: Getting Training to Perform Well	328 14:39	DL 8.7	nn Sect. 5
Neural Networks: Smoothing (to Motivate Convolution)	330 10:17		nn Sect. 6
Neural Networks: Convolution	332 12:53		nn Sect. 6
Convolutional Neural Networks: Introduction	334 12:10	DL 9.2	nn Sect. 7
Convolutional Neural Networks: Forward Propagation	336 17:08		nn Sect. 7.7
Convolutional Neural Networks: Demonstration	338 7:17		nn Sect. 7.7, 7.8	+
Exercises: How would you extend our notation for forward propagation in a convolutional neural network (1) for 2D convolution and (2) including bias units?
2021-01-12Convolutional and Autoencoder Networks
Convolutional Neural Networks: Pooling; Typical Architecture	340 6:05		nn Sect. 7.9–7.12
Convolutional Neural Networks: Example Architecture and Features	342 9:05		nn Sect. 7.13, 7.14
Convolutional Neural Networks: Error Backpropagation	344 9:24		nn Sect. 7.15, 7.16	+
Exercises: Using the chain rule for partial derivatives, show that \(\nabla_{\mathbf{w}_{k g}^{(l)}} E(W, \mathbf{b}; \mathbf{x}, \mathbf{t}) = \mathbf{\delta}_k^{(l+1)} * \tilde{\mathbf{a}}_g^{(l)}\).
Convolutional Neural Networks: Variants of Connectivity	346 6:07		nn Sect. 8
Neural Networks: Sparse Autoencoder; Kullback-Leibler Divergence	350 12:18		nn Sect. 9	+
Exercises: Why does a low target activation probability encourage sparsity; why doesn’t it simply lead to correspondingly low activation levels throughout? Derive the result for \(\delta_j^{(2)}\).
Neural Networks: Features learned by a sparse autoencoder	352 6:08		nn Sect. 9.5	+
Exercises: Whitening serves to normalize the mean and variance of the intensities of the images. What purpose does it serve here?
2021-01-19Kernel Methods
Kernel Methods		PRML 6–6.2	PRML-06 Sect. 2, 3, 5.1	+
Exercises: See the notes.

RSS Feed

Justus Piater, 2020/10/11 22:28

The preparation material for 2020-10-13 has been finalized.

Justus Piater, 2020/10/15 21:02

The preparation material for 2020-10-20 has been finalized.

Sayantan Auddy, 2020/10/22 23:34

The preparation material for 2020-10-27 has been finalized.

Sayantan Auddy, 2020/10/28 12:59

The preparation material for 2020-11-03 has been finalized.

Justus Piater, 2020/10/28 14:22

This page is now accessible from the internet; a VPN is no longer required.

Sayantan Auddy, 2020/11/06 17:57

The preparation material for 2020-11-10 has been finalized.

Sayantan Auddy, 2020/11/14 02:35

The preparation material for 2020-11-17 has been finalized.

Justus Piater, 2020/11/21 22:03

The preparation material for 2020-11-24 has been finalized.

Justus Piater, 2020/11/29 09:51, 2020/11/29 09:53

The preparation material for 2020-12-01 has been finalized. Please do the exercises before class!

Justus Piater, 2020/12/11 17:45

The preparation material for 2020-12-15 has been finalized. Please always do the exercises before class!

Justus Piater, 2021/01/10 20:23

The preparation material for 2021-01-12 has been finalized.

Justus Piater, 2021/01/16 13:15

The preparation material for 2021-01-19 has been finalized.

Please read the given sections in the book, try to follow the math, and give the exercises referred to in the course notes a shot. We will look at these on Tuesday.

Sidebar

Table of Contents

Advanced Machine Learning (VO2 + PS1, Master)

Course Concept

Textbooks

Agenda

RSS Feed

User Tools

Site Tools

Sidebar

Table of Contents

Advanced Machine Learning (VO2 + PS1, Master)

Course Concept

Textbooks

Agenda

RSS Feed

Page Tools