Intelligent and Interactive Systems

Generative KI-Systeme wie ChatGPT und Midjourney haben die Welt im Sturm erobert. Wie können wir sie produktiv nutzen? Wie können wir mit den Herausforderungen umgehen, die durch sie entstehen? Ich werde diese Fragen von der technischen Seite her angehen und einen Eindruck davon vermitteln, wie diese Systeme funktionieren. Daraus ergeben sich ein realistisches Verständnis für ihre Möglichkeiten und Grenzen sowie einige grundlegende Empfehlungen für den Umgang mit ihnen im Schulbetrieb.

Reinforcement learning is a promising technique for solving complex control problems in real-world physical systems, such as robotics, plasma stabilization, and particle accelerators. However, RL is often data-hungry, and its classic on-policy formulation is often inefficient, as it disallows data reuse, and unsafe, as it requires the agent to interact with the environment from scratch. Off-policy reinforcement learning offers a more appealing paradigm by enabling the reuse of historical data and the utilization of safe, external behavior sources (such as human operator logs). However, this flexibility comes at a cost: off-policy learning introduces significant theoretical instabilities. In this talk, we will analyze some fundamental difficulties in off-policy reinforcement learning, both in value and policy learning, explore the algorithmic landscape that tames them, and see the future direction in which the field is moving.

AI has made great progress in recent years, and the sophistication of robots has been rising with costs falling. Yet, the capabilities of AI-enabled robots are not keeping pace. I argue that this is due to a lack of structural understanding by current AI systems. I will discuss several lines of research in my lab that seek to enable robots to generalize better and learn faster thanks to explicit notions of structure.

As artificial intelligence reshapes our digital world at a breathtaking pace, a curious question arises: Where are all the real-world, intelligent robots? While we have mastered the generation of text, images, and video by leveraging vast web-scale datasets, robotics still faces a fundamental data bottleneck. Reinforcement learning (RL) offers a compelling solution, enabling agents to learn autonomously by collecting their own experience. However, the path to autonomy is often blocked by the staggering inefficiency of current RL algorithms, which can require millions of trials to master simple tasks. This talk embarks on a quest to tackle this efficiency problem head-on. I will argue that a crucial step toward unlocking the potential of robot learning lies in a two-pronged approach: first, by developing statistically efficient algorithms that can reuse data by leveraging sound off-policy techniques, and second, by designing better action representations for physical, real-world agents. By making our algorithms more efficient and refining their core hypotheses, we can accelerate the journey toward real embodied artificial intelligence.

The flexibility and robustness of current robots is limited by their lack of understanding of their environment. For this reason, most robots operate in controlled environments. Machine learning can circumvent modeling problems but introduces new problems of generalizing from examples. How can robots acquire understanding (of structure, function, causality, etc.) that allows them to generalize from sparse experience? Motivated by shortcomings of current machine-learning methods, I will argue that "understanding" is a meaningful notion in AI that reaches beyond prediction and control. I will discuss examples of our recent work on learning visual relational concepts, extrapolation of learned movements beyond the training distribution, learning of symbolic concepts and rules, and structure-driven skill learning from sensorimotor experience. Our long-term objective is to improve abstraction, generalization, robustness, and ultimately explainability of robot perception and action.