research
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
research [2014/07/29 14:18] c7031018 |
research [2018/09/03 19:35] (current) |
||
|---|---|---|---|
| Line 12: | Line 12: | ||
| The following list highlights some examples of our current and past work. Follow the links for more information. | The following list highlights some examples of our current and past work. Follow the links for more information. | ||
| + | |||
| + | {{ :research:visualcortex.png?nolink&260|Areas of the visual cortex}} **[[https://iis.uibk.ac.at/public/antonio/Research.html|Computational neuroscience]]** - Due to the complexity of vision tasks (object recognition, motion and stereo analysis) for computers, many scientists and engineers have resorted towards neurophysiology for solutions on how the human visual system solves such difficult tasks with astonishing efficiency and accuracy. Recent approaches to object recognition are mainly driven by the "edge doctrine" of visual processing pioneered by Hubel and Wiesel's work (that led to their 1981 Nobel Prize in Medicine). Edges - as detected by responses of simple and complex cells - provide important information about the presence of shapes in visual scenes. We consider that their detection is only a first step at generating an interpretation of images. Our line of work focuses on intermediate level processing areas, these operate upon initial simple and complex cell outputs towards the formation of neural representations of scene content that allow robust shape inference and object recognition. \\ Check our PLOS ONE publications in [[http://dx.doi.org/10.1371/journal.pone.0042058|Rodriguez-Sanchez and Tsotsos (2012)]] and [[http://dx.doi.org/10.1371/journal.pone.0098424|Azzopardi, Rodriguez-Sanchez, Piater and Petkov (2014)]] as well as our [[http://dx.doi.org/10.1109/TPAMI.2012.272|summary PAMI paper]]. | ||
| + | |||
| + | <html><div style="clear:both"></div><br></html> | ||
| + | |||
| + | |||
| + | <html> | ||
| + | <div style="border:0;float:right;margin:0 0 0 1.5em"> | ||
| + | <video width="280" height="170" controls preload="metadata"> | ||
| + | <source src="/public/videos/pacman_demo.ogg" type='video/ogg;codecs="theora, vorbis"'> | ||
| + | <applet code="com.fluendo.player.Cortado.class" archive="/public/cortado.jar" width="280" height="170"> | ||
| + | <param name="autoPlay" value"false" /> | ||
| + | <param name="url" value="/public/videos/pacman_demo.ogg"/> | ||
| + | </applet> | ||
| + | </video> </div> | ||
| + | </html> **Multimodal, hierarchical models for object manipulation** - We have developed a generic framework for object grasping using our robotic platform. The main software modules perform object detection and pose estimation, grasp planning, path planning and robot arm and hand trajectory execution. Over the course of the [[http://www.pacman-project.eu/|PaCMan project]], we will replace many of these modules by advanced modules developed by the project. For example, the object detection component, currently based on state-of-the-art methods, will benefit from hierarchical compositional models based on both 2D and 3D information. This representation will be useful both for reasoning in a cluttered environment where only parts of an object will be visible, and for improving object manipulation. | ||
| + | |||
| + | <html><div style="clear:both"></div><br></html> | ||
| + | |||
| + | <html> | ||
| + | <div style="border:0;float:right;margin:0 0 0 1em"> | ||
| + | <video width="270" height="180" controls preload="metadata"> | ||
| + | <source src="/public/videos/symbol-formation.ogg" type='video/ogg;codecs="theora, vorbis"'> | ||
| + | <applet code="com.fluendo.player.Cortado.class" archive="/public/cortado.jar" width="280" height="170"> | ||
| + | <param name="autoPlay" value"false" /> | ||
| + | <param name="url" value="/public/videos/symbol-formation.ogg"/> | ||
| + | </applet> | ||
| + | </video> </div> | ||
| + | </html> | ||
| + | **[[https://iis.uibk.ac.at/public/emre/research.html|From Continuous Manipulative Exploration to Symbolic Planning]]** - This work aims for bottom-up and autonomous development of symbolic planning operators from continuous interaction experience of a manipulator robot that explores the environment using its action repertoire. In the first stage, the robot explores the environment by executing actions on single objects, forms effect and object categories, and gains the ability to predict the object/effect categories from the visual properties of the objects by learning the nonlinear and complex relations among them. In the next stage, with further interactions that involve stacking actions on pairs of objects, the system learns logical high-level rules that return a stacking-effect category given the categories of the involved objects and the discrete relations between them. Finally, these categories and rules are encoded in PDDL format, enabling symbolic planning. In the third state, the robot progressively updates the previously learned concepts and rules in order to better deal with novel situations that appear during multi-step plan executions. This way, categories of novel objects can be inferred or new categories can be formed based on previously learned rules. Our system further learns probabilistic rules that predict the action effects and the next object states. After learning, the robot was able to build stable towers in real world, exhibiting some interesting reasoning capabilities such as stacking larger objects before smaller ones, and predicting that cups remain insertable even with other objects inside. ([[https://iis.uibk.ac.at/public/emre/papers/ICRA2015.pdf|ICRA2015.pdf]], [[https://iis.uibk.ac.at/public/emre/papers/humanoids.pdf|humanoids.pdf]]). | ||
| + | |||
| + | <html><div style="clear:both"></div><br></html> | ||
| + | |||
| + | <html> | ||
| + | <div style="border:0;float:right;margin:0 0 0 1em"> | ||
| + | <video width="270" height="180" controls preload="metadata"> | ||
| + | <source src="/public/videos/bootstrapping.ogg" type='video/ogg;codecs="theora, vorbis"'> | ||
| + | <applet code="com.fluendo.player.Cortado.class" archive="/public/cortado.jar" width="280" height="170"> | ||
| + | <param name="autoPlay" value"false" /> | ||
| + | <param name="url" value="/public/videos/bootstrapping.ogg"/> | ||
| + | </applet> | ||
| + | </video> </div> | ||
| + | </html> | ||
| + | **[[https://iis.uibk.ac.at/public/emre/research.html|Bootstrapped learning and Emergent Structuring of interdependent single and multi-object affordances]]** - Inspired from infant development, we propose a learning system for a developmental robotic system that benefits from bootstrapping, where learned simpler structures (affordances) that encode robot's interaction dynamics with the world are used in learning of complex affordances ([[https://iis.uibk.ac.at/public/emre/papers/ICDL2014-Bootstrapping.pdf|ICDL2014-Bootstrapping]]). In order to discover the developmental order of different affordances, we use Intrinsic Motivation approach that can guide the robot to explore the actions it should execute in order to maximize the learning progress. During this learning, the robot also discovers the structure by learning and using the most distinctive object features for predicting affordances. The results show that the hierarchical structure and the development order emerged from the learning dynamics that is guided by Intrinsic Motivation mechanisms and distinctive feature selection approach ([[https://iis.uibk.ac.at/public/emre/papers/ICDL2014-EmergentStructuring.pdf|ICDL2014-EmergentStructuring.pdf]]). | ||
| + | |||
| + | <html><div style="clear:both"></div><br></html> | ||
| {{ :research:teney-2013-crv4.jpg?nolink&240|Probabilistic models of object appearance}} **[[:research:appearance-models|Probabilistic models of appearance]] for object recognition and pose estimation in 2D images** - We developed methods to represent the appearance of objects, and associated inference methods to identify them in images of cluttered scenes. The goal here is to leverage, to a maximum, the information conveyed by 2D images alone, without resorting to stereo or other 3D sensing techniques. We are also interested in recovering the precise pose (3D orientation) of objects, so as to ultimately use such information in the context of robotic interaction and grasping. | {{ :research:teney-2013-crv4.jpg?nolink&240|Probabilistic models of object appearance}} **[[:research:appearance-models|Probabilistic models of appearance]] for object recognition and pose estimation in 2D images** - We developed methods to represent the appearance of objects, and associated inference methods to identify them in images of cluttered scenes. The goal here is to leverage, to a maximum, the information conveyed by 2D images alone, without resorting to stereo or other 3D sensing techniques. We are also interested in recovering the precise pose (3D orientation) of objects, so as to ultimately use such information in the context of robotic interaction and grasping. | ||
| Line 40: | Line 86: | ||
| {{ :research:pdm-tracking.jpg?nolink&200|}} **[[:research:object-tracking|Real-Time Object Tracking]] in Complex Scenes** - While most work at the time was based on background subtraction, we developed new methods for complex scenes by tracking local features for robustness to occlusions and to background changes, taking spatial coherence into account for robustness to overlapping, similar-looking targets. In the context of soccer, we also developed methods for robust, model-based and model-free, absolute and incremental terrain tracking. | {{ :research:pdm-tracking.jpg?nolink&200|}} **[[:research:object-tracking|Real-Time Object Tracking]] in Complex Scenes** - While most work at the time was based on background subtraction, we developed new methods for complex scenes by tracking local features for robustness to occlusions and to background changes, taking spatial coherence into account for robustness to overlapping, similar-looking targets. In the context of soccer, we also developed methods for robust, model-based and model-free, absolute and incremental terrain tracking. | ||
| - | [[https://iis.uibk.ac.at/public/antonio/Research.html|Computational neuroscience]] Object recognition is a very hard problem, the best proof of this is to consider that the first works started in the 1960s (Robert's and Guzman's theses) and up to now there are thousands of papers published each year with solutions to achieve that goal. Due to the complexity of this problem, many scientists and engineers have resorted towards neurophysiology for solutions on how the human visual system solves such a difficult task with astonishing efficiency and accuracy. The earliest models inspired by the primate visual system appeared little after the influential works of Hubel and Wiesel (1962,1965,1968) that revealed key mechanisms of the functional architecture of the visual cortex. Computational neuroscience deals with the simulation of the different functions of biological neurons by approximating that functionality by means of mathematical approximations. The recent approaches to object recognition are mainly driven by the ''edge doctrine'' of visual processing pioneered by Hubel and Wiesel's work. Edges, as detected by responses of simple and complex cells, provide important information about the presence of shapes in visual scenes. We consider that their detection, however, is only a first step at generating an interpretation of images that is invariant against certain variances in scene properties and layout as well as their geometric transformations. Thus, our line of work is on intermediate processing. Intermediate processing areas operate upon initial simple and complex cell outputs lead to the formation of neural representations of scene content that allow robust shape inference and object recognition. We focus on the steps that computer models in our opinion must pursue in order to develop robust recognition mechanisms that mimic biological processing capabilities beyond the level of cells with classical simple and complex receptive field response properties. Check our publications [[http://dx.doi.org/10.1371/journal.pone.0042058|Rodriguez-Sanchez and Tsotsos, 2012]] and [[http://dx.doi.org/10.1371/journal.pone.0098424|Rodriguez-Sanchez et al, 2014]]. | + | <html><div style="clear:both"></div><br></html> |
| + | |||
research.1406636339.txt.gz · Last modified: 2018/09/03 14:57 (external edit)