IMAGINE - Robots Understanding Their Actions by Imagining Their Effects (EU H2020, 2017-2020): seeks to enable robots to understand the structure of their environment and how it is affected by its actions. “Understanding” here means the ability of the robot (a) to determine the applicability of an action along with parameters to achieve the desired effect, and (b) to discern to what extent an action succeeded, and to infer possible causes of failure and generate recovery actions.
SQUIRREL (EU FP7-ICT-STREP, 2014-2018): Clutter in an open world is a challenge for many aspects of robotic systems, especially for autonomous robots deployed in unstructured domestic settings, affecting navigation, manipulation, vision, human robot interaction and planning. SQUIRREL addresses these issues by actively controlling clutter and incrementally learning to extend the robot's capabilities while doing so. We term this the B3 (bit by bit) approach, as the robot tackles clutter one bit at a time and also extends its knowledge continuously as new bits of information become available. SQUIRREL is inspired by a user driven scenario, that exhibits all the rich complexity required to convincingly drive research, but allows tractable solutions with high potential for exploitation. We propose a toy cleaning scenario, where a robot learns to collect toys scattered in loose clumps or tangled heaps on the floor in a child's room, and to stow them in designated target locations.
3rdHand (EU FP7-ICT-STREP, 2013-2017) develops a semi-autonomous robot assistant that acts as a third hand of a human worker. It will be straightforward to instruct even by an untrained layman worker, allow for efficient knowledge transfer between tasks, and enable effective collaboration between a human worker with a robot third hand. The main contributions of this project will be the scientific principles of semi-autonomous human-robot collaboration, a new semi-autonomous robotic system that is able to (i) learn cooperative tasks from demonstration, (ii) learn from instruction, and (iii) transfer knowledge between tasks and environments.
PaCMan (EU FP7-ICT-STREP, 2013-2016) advances methods for object perception, representation and manipulation so that a robot is able to robustly manipulate objects even when those objects are unfamiliar, and even though the robot has unreliable perception and action. The proposal is founded on two assumptions. The first of these is that the representation of the object's shape in particular and of other properties in general will benefit from being compositional (or very loosely hierarchical and part based). The second is that manipulation planning and execution benefits from explicitly reasoning about uncertainty in object pose, shape etcetera; how it changes under the robot's actions, and the robot should plan actions that not only achieve the task, but gather information to make task achievement more reliable.
Xperience (EU FP7-ICT-IP, 2011-2015) pursues two principal objectives. The first goal is to show that the state of the art enactive embodied cognition systems can be significantly enhanced by using structural bootstrapping - a concept taken from language learning. The second goal is to implement a complete robot system for automating introspective, predictive, and interactive understanding of actions and dynamic situations.
IntellAct (EU FP7-ICT-STREP, 2011-2014) addresses the problem of understanding and exploiting the meaning (semantics) of manipulations in terms of objects, actions and their consequences for reproducing human actions with machines. This is in particular required for the interaction between humans and robots in which the robot has to understand the human action and then to transfer it to its own embodiment.
LearnBiP (EU FP7-ICT ECHORD Experiment, 2011-2012) has two main aims. First it utilizes the huge amount of data generated in industrial bin-picking for the introduction of grasp learning. Second it evaluates the potential of the SCHUNK dexterous hand SDH-2 for its application in industrial bin-picking.
SignSpeak (EU FP7-ICT-STREP, 2009-2012) focused on scientific understanding and vision-based technological development for continuous sign language recognition and translation. The aim was to increase the linguistic understanding of sign languages and to create methods for transcribing sign language into text. See an article in the Projects magazine.
PACO-PLUS (“perception, action and cognition through learning of object-action complexes”, EU FP6-IST-IP, 2006-2010) brings together an interdisciplinary research team to design and build cognitive robots capable of developing perceptual, behavioural and cognitive categories that can be used, communicated and shared with other humans and artificial agents. To demonstrate our approach we are building robot systems that will display increasingly advanced cognitive capabilities over the course of the programme. They will learn to operate in the real world and to interact and communicate with humans. To do this they must model and reflectively reason about their perceptions and actions in order to learn, act and react appropriately. We hypothesize that such understanding can only be attained by embodied agents and requires the simultaneous consideration of perception and action.
TRICTRAC (2003-2006), directed by J. Piater, aimed at the development of algorithms for real-time object tracking in one or more live video streams. It was a joint project between the Université de Liège and the Université Catholique de Louvain funded by the Walloon Region. Some results are summarized in a video.