Intelligent and Interactive Systems

User Tools

Site Tools


courses:2020w:703644:start

RSS

Probabilistic Models and Inference (VO1 + PS1, Master)

See the UIBK course catalog entries for the lecture and the seminar for essential information.

This course will be organized similarly to its companion course Advanced Machine Learning (VO2 + PS1, Master). The online sessions will last about one hour and will encompass VO discussion and the PS; thus, attendance is mandatory for all PS registrants.

There is an online forum for students to ask and answer each other's questions, where also the instructors will answer questions.

Some Suggested Topics

  • Boltzmann Machines; Deep Belief Networks
  • Hidden Markov Models
  • Density Estimation
  • Sampling Techniques; MCMC
  • Latent variable models
  • Variational Inference (vs. MCMC; Variational Auto Encoder)

Suggest more and/or vote for any of these.

Texts

  • DE Density Estimation for Statistics and Data Analysis by B.W. Silverman. We only refer to the first two chapters.
  • Doersch and K&W tutorials on variational autoencoders: see the VI notes.

Agenda

Homework Assignments

Programming assignments are to be downloaded from JupyterHub, solved on your local machine, and reuploaded to JupyterHub, exactly like in the AdvML PS.

  1. ProbMod-01-linear-regression is due 2020-11-11.
  2. ProbMod-02-factor-graph is due 2021-01-13.
  3. ProbMod-03-mean-field is due 2021-01-27.

Lectures

In addition to the resources given below, this course extensively uses Bishop's slides for PRML chapters 1 and 8.

The videos are in OLAT.

Topic Video Text Notes Ex.
2020-10-08Linear Regression
Introduction intro
Linear Regression PRML 1.1 PRML-01
2020-10-15Probability Theory
Probability Theory: Random Variables 8 5:13 PRML-01 +
Probability Theory: Probability Densities 10 7:21 PRML 1.2.1 PRML-01 +
Probability Theory: Expectations 12 9:24 PRML 1.2.2 PRML-01 +
Probability Theory: Covariance 14 14:19 PRML 1.2.2 PRML-01 +
2020-10-22ML and MAP Estimation; Probabilistic Regression
Probability Theory: Normal Distribution 24 11:07 PRML 1.2.4 PRML-01 +
Probability Theory: Maximum-Likelihood Estimation of the Paramters of a Normal Distribution 26 17:39 PRML 1.2.5 PRML-01 +
Probability Theory: ML Estimate of Variance is Biased 28 4:38 PRML 1.2.4 PRML-01 +
Regression: Maximum-Likelihood Parameter Estimation 30 14:39 PRML 1.2.5 PRML-01 +
Regression: Maximum A Posteriori Parameter Estimation 32 12:13 PRML 1.2.5 PRML-01 +
2020-10-29Probabilistic Prediction; Classification
Regression: Bayesian Prediction 34 13:11 PRML 1.2.6 PRML-01 +
Probabilistic Prediction: ML and MAP Parameter Estimation; Bayesian Prediction 36 8:06 PRML 1.2.6 PRML-01 +
Classification: ML and MAP 40 11:24 PRML 1.5–1.5.1; 1.5.3 PRML-01 +
Classification: Generative and Discriminative Models; Reject Option 42 7:48 PRML 1.5.3–1.5.4 PRML-01 +
2020-11-05Classification: Loss; Disciminative Models
Classification: Loss Matrices, Loss Functions 44 12:57 PRML 1.5.2 PRML-01 +
Classification: High-Level Comparison of Probabilistic and Non-Probabilistic Paradigms 46 4:16 PRML-01 +
Discriminative Models for Classification: Linear Discriminants 50 10:33 PRML 4.2–4.2.1 PRML-04 +
Discriminative Models for Classification: Nonlinear Discriminants; the Exponential Family 52 4:16 PRML 4.2.1 PRML-04 +
Discriminative Models for Classification: Basis Functions 54 9:25 PRML 4.3.1, 4.3.2 PRML-04
Discriminative Models for Classification: Maximum-Likelihood 56 11:58 PRML 4.3.2 PRML-04 +
Discriminative Models for Classification: Logistic Regression for 2 Classes 58 13:28 PRML 4.3.3 PRML-04 +
Discriminative Models for Classification: Logistic Regression for K Classes 60 5:18 PRML 4.3.4 PRML-04 +
2020-11-12Bayesian Networks: Factorization; Conditional Independence
Bayesian Networks: Factorization and Corresponding Directed Acyclic Graph 100 11:00 PRML 8.1 (Intro) PRML-08 +
Bayesian Networks: Plate Notation; Modeling Regression 102 10:28 PRML 8.1.1 PRML-08 +
Bayesian Networks: Counting Model Parameters 104 9:05 PRML 8.1.3 +
Bayesian Networks: Conditional Independence 106 4:49 PRML 8.2 (Intro) PRML-08 +
Bayesian Networks: Conditional Independence: Tail-to-Tail and Head-to-Tail Connections 108 10:21 PRML 8.2.1
Bayesian Networks: Conditional Independence: Head-to-Head Connection 110 6:29 PRML 8.2.1 PRML-08 +
Bayesian Networks: Fuel Gauge Example; Explaining Away PRML 8.2.1 +
2020-11-19BN: d-Separation and Markov Blanket; MRF: Conditional Indpendence Properties
Bayesian Networks: d-Separation 114 9:55 PRML 8.2.2 PRML-08 +
Bayesian Networks: d-Separation examples: IID variables; Naive Bayes classifier 116 14:35 PRML 8.2.2 PRML-08
Bayesian Networks: Metaphor of filtering by factorization or by d-separation 118 2:59 PRML 8.2.2 +
Bayesian Networks: Markov Blanket 120 5:55 PRML 8.2.2 +
Markov Random Fields: Conditional Independence Properties PRML 8.3.1 +
2020-11-26MRF: Factorization; Relations to BN; Inference on Chains and Trees
Markov Random Fields: Factorization Properties PRML 8.3.2, 8.3.3 +
Graphical Models: Relations Between Directed and Undirected Models PRML 8.3.4 +
Markov Chain: Marginalization Using Distributive Law 128 9:58 PRML 8.4.1
Markov Chain: Marginalization Using Distributive Law: Example 130 8:16 PRML-08 +
Markov Chain: Marginalization Using Message Passing 132 5:08 PRML 8.4.1 +
Markov Chain: Marginalization Using Message Passing: Example 134 5:57 PRML-08
Markov Chain: Marginalization For All Variables 136 2:58 PRML 8.4.1
MRF Trees PRML 8.4.2 +
2020-12-03Factor Graphs; Sum-Product Algorithm
Factor Graphs PRML 8.4.3 +
Sum-Product Algorithm PRML 8.4.4 PRML-08 Sect. 5, 6, 9.3 +
2020-12-10Max-Product Algorithm; General Graphs; Loopy Belief Propagation
Max-Sum Algorithm PRML 8.4.5 PRML-08 Sect. 7, 8, 9.4 +
Exact Inference in General Graphical Models PRML 8.4.6
Loopy Belief Propagation PRML 8.4.7
2020-12-17Kernel Density Estimation
Kernel Density Estimation: Motivation 400 7:49 DE 1 KDE Sect. 2
Kernel Density Estimation: Histogram 402 8:01 DE 2.2 KDE Sect. 2.6
Kernel Density Estimation: Naive Estimator 404 6:56 DE 2.3 KDE Sect. 2.7, 2.8
Kernel Density Estimation: Kernel Estimator 406 3:45 DE 2.4 KDE Sect. 3.1, 3.2
Kernel Density Estimation: Nearest-Neighbor Estimator 408 5:53 DE 2.5 KDE Sect. 3.3, 3.4
Kernel Density Estimation: Variable-Kernel Estimator; General Remarks 410 4:52 DE 2.6 KDE Sect. 3.5, 4
Kernel Density Estimation: Bounded Domains 412 6:50 DE 2.10 KDE Sect. 5
Kernel Density Estimation: Circular Domains 414 4:04 KDE Sect. 5
2021-01-07Hidden Markov Models
Hidden Markov Models HMM +
2021-01-14Variational Inference
Variational Inference vi Sect. 13 +
2021-01-21Variational Autoencoders
Variational Autoencoders Doersch 1–2.3; K&W 1–2.5 (Intro); 2.6 vi Sect. 4 +
2021-01-28Markov-Chain Monte Carlo
Markov-Chain Monte Carlo PRML 11.2–11.3 MCMC +

RSS Feed

Justus Piater, 2020/10/11 22:27

The preparation material for 2020-10-15 has been finalized.

Justus Piater, 2020/10/19 20:50

The preparation material for 2020-10-22 has been finalized.

Justus Piater, 2020/10/26 21:40, 2020/10/28 07:47

The preparation material for 2020-10-29 has been finalized.

Justus Piater, 2020/10/28 14:23

This page is now accessible from the internet; a VPN is no longer required.

Justus Piater, 2020/11/02 17:25

The first programming assignment is out.

Justus Piater, 2020/11/02 23:41

The preparation material for 2020-11-05 has been finalized.

Justus Piater, 2020/11/07 18:18

Assignment 1 Task 3 Opening Paragraph

Write a function that computes the predictive distribution expressed by the vectors $\hat{\mu}$ and $\hat{\sigma}^2$ for a set of test input values, received in the form of its design matrix $\Phi_{\textrm{test}}$, given the design matrix $\Phi$ and target values $\mathbf{t}$ of the training data, and the meta-parameter $\alpha$.

Justus Piater, 2020/11/08 11:48

The preparation material for 2020-11-12 has been finalized.

Justus Piater, 2020/11/12 16:59

Assignment 1 has been graded and feedback released.

Justus Piater, 2020/11/16 15:55

The preparation material for 2020-11-19 has been finalized.

Justus Piater, 2020/11/22 20:17

The preparation material for 2020-11-26 has been finalized.

Justus Piater, 2020/12/02 08:47

The preparation material for 2020-12-03 has been finalized.

Justus Piater, 2020/12/07 19:45

The preparation material for 2020-12-10 has been finalized.

Justus Piater, 2020/12/12 18:42

The preparation material for 2020-12-17 has been finalized.

Justus Piater, 2020/12/19 21:56

The second programming assignment is out.

Justus Piater, 2021/01/14 19:28, 2021/01/17 21:02

Oral exams will take place February 4 online. Please register by January 21.

Online exams are subject to the conditions Absolvierung der kommissionellen Abschlussprüfung bzw. Defensio. In addition:

  • You need to be able to write and draw in a way that I can see it on my screen. Thus, you need either a tablet (possibly separately connected to the video call), a webcam facing a sheet of paper or a board, or equivalent. Please arrange and test this beforehand!
  • I will occasionally take screenshots of the video call for my records (no video, no voice).
Justus Piater, 2021/01/15 19:46

The third (and last) programming assignment is out.

Justus Piater, 2021/01/18 10:24

The preparation materials for 2021-01-21 have been finalized. Please read the given sections from the two tutorials as explained in the lecture notes. In class we will go over the entire material again following the lecture notes, but to follow it will be essential to have read the texts beforehand.

Justus Piater, 2021/01/22 15:44

The oral exam schedule can be found in OLAT.

Justus Piater, 2021/01/22 17:52

I released the results of Assignment 2. I manually checked and verified all failed autograder tests.

Justus Piater, 2021/01/24 17:12

The preparation material for 2020-01-28 has been finalized.

Justus Piater, 2021/02/04 22:02

I released the results of Assignment 3.

xffrebplpn, 2021/03/22 04:46

Muchas gracias. ?Como puedo iniciar sesion?

Justus Piater, 2021/03/23 12:03

The next round of oral exams will take place May 4th. Please register by April 20. They will take place online, under the same conditions as the first round.

Justus Piater, 2021/05/10 16:08

The next round of oral exams will take place September 24. Please register by September 10. They will either take place online under the same conditions as the first round, or in presence in ICT 3W06.

Justus Piater, 2021/09/11 20:25

The oral exam schedule can be found in OLAT.

Justus Piater, 2021/10/08 09:28, 2021/10/08 09:34

The next round of oral exams will take place December 14.

Enter your comment. Wiki syntax is allowed:
 
courses/2020w/703644/start.txt · Last modified: 2021/01/18 10:17 by Justus Piater