Action Representations in Robotics.

Abstract: Understanding and defining the meaning of ``action'' is substantial for robotics research. This becomes utterly evident when aiming at equipping autonomous robots with robust manipulation skills for action execution. Unfortunately, to this day we still lack both a clear understanding of the concept of an action and a set of established criteria that ultimately characterize an action. In this survey we thus first review existing ideas and theories on the notion and meaning of action. Subsequently we discuss the role of action in robotics and attempt to give a seminal definition of action in accordance with its use in robotics research. Given this definition we then introduce a taxonomy for categorizing action representations in robotics along various dimensions. Finally, we provide a systematic literature survey on action representations in robotics where we categorize relevant literature along our taxonomy. After discussing the current state of the art we conclude with an outlook towards promising research directions.

Original publication: IJRR (open access)

Download the data as MS Excel file (50 KB): Download»


Title Author Year Perspective Stimuli Selective Attention Granularity Abstraction Competition Sequencing Generalization Motivation Acquisition Prediction Exploitation Learning Discretization Grounding Associativity Effect Correspondence Formulation Method Features Training Evaluation Datasets Actions Date Added
Title Author Year Perspective Stimuli Selective Attention Granularity Abstraction Competition Sequencing Generalization Motivation Acquisition Prediction Exploitation Learning Discretization Grounding Associativity Effect Correspondence Formulation Method Features Training Evaluation Datasets Actions Date Added
Teaching new tricks to a robot learning to solve a task by imitationAcosta et al.2010observerextroceptiveyesmesoatomicnoyesyesextrinsicDemonstrationclassificationSingle-/Multi-step predictionofflinebothyesbidirectionalbothmathematicalForward and Inverse Models, FSMmarker posessupervisedReal Robot pick and placeAug 2018
Toward a library of manipulation actions based on semantic object-action relationsAein et al.2013agentbothnomesoatomicnonoyesnot specifiedHard codedclassificationSingle-/Multi-step predictionofflinecategoricalyesunidirectionalbothmathematicalSEC, FSM, DMPobject and gripper posessupervisedReal Robot pushing, reaching, grasping, hidingAug 2018
Action recognition by employing combined directional motion history and energy imagesAhad et al.2010observerextroceptivenoglobalatomicnonoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalNNcartesian moments, energy templatesunsupervisedBenchmark wave, bend, hug, jumpAug 2018
Variable silhouette energy image representations for recognizing human actionsAhmad & Lee2010observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalMCSVMsilhouette images and shapesupervisedBenchmarkKTH, FBG Aug 2018
Learning symbolic representations of actions from human demonstrationsAhmadzadeh et al.2015agentbothnomesoatomicnoyesyesnot specifiedDemonstrationoptimizationPlanningofflinebothyesunidirectionalbothmathematicalDMPs, Visuospatial Skill Learningtrajectories, object motionssupervisedReal Robot pull, pushAug 2018
Enriched manipulation action semantics for robot execution of time constrained tasksAksoy et al.2016agentbothnomesoatomicnonoyesnot specifiedDemonstrationclassificationSingle-/Multi-step predictionofflinecontinuousyesunidirectionalbothmathematicalMMM, SECobject and body posessupervisedReal Robot manipulation, interactionAug 2018
Structural bootstrapping at the sensorimotor level for the fast acquisition of action knowledge for cognitive robotsAksoy et al.2013observerbothyesmesoatomicnonoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalyesunidirectionalbothmathematicalSEC, DMP, Structural Bootstrappingobject and hand posessupervisedBenchmark cutting, stirring, choppingAug 2018
Semantic Decomposition and Recognition of Long and Complex Manipulation Action SequencesAksoy et al.2016observerextroceptiveyesmesobothyesyesyesnot specifiedDemonstrationoptimizationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalSEC, Similarity Matchingdepth, colorunsupervisedBenchmarkManiAc Aug 2018
Deep Feature-Action Processing with Mixture of UpdatesAltahhan2015agentextroceptiveyesglobalatomicyesyesyesintrinsicExplorationregressionSingle-/Multi-step predictiononlinecontinuousyesunidirectionalbodymathematicalPCA, Actor-CriticRGB imagessupervisedReal Robot homingAug 2018
Learning Invariant Sensorimotor Behaviors: A Developmental Approach to Imitation MechanismsAndry et al.2004agentbothyesmesoatomicnonoyesnot specifiedDemonstrationclassificationSingle-/Multi-step predictiononlinecontinuousnounidirectionalenvironmentmathematicalNF, DS, SOMjoint configurationsunsupervisedReal Robot pointing, navigation, gesturesAug 2018
Perceiving Objects and Movements to Generate Actions on a Humanoid RobotAsfour et al.2008observerextroceptiveyesglobalatomicnoyesyesnot specifiedcombinationregressionSingle-/Multi-step predictionofflinecategoricalnounidirectionalenvironmentmathematicalParticle filter, HMM, Clusteringinvariant object features, affordancesunsupervisedReal Robot reaching, grasping, placingAug 2018
Human sensorimotor learning for humanoid robot skill synthesisBabic et al.2011agentextroceptivenoglobalatomicnonoyesnot specifiedDemonstrationoptimizationSingle-/Multi-step predictionofflinecontinuousnounidirectionalbodybiomimeticRBF networkjoint configurationssupervisedReal Robot reachingAug 2018
Constraint-based movement representation grounded in geometric featuresBartls et al.2013agentextroceptivenomesoatomicnoyesyesnot specifiedHard codedoptimizationPlanningofflinecategoricalnounidirectionalenvironmentmathematicalTask functionspoints, lines, planessupervisedReal Robot making pancakeAug 2018
Representing robot/environment interactions using probabilities: the "beam in the bin" experimentBessiere et al.1994agentbothnoglobalatomicnonoyesnot specifiedExplorationinferenceSingle-/Multi-step predictiononlinecontinuousyesbidirectionalbothmathematicalProbabilistic Inferencemotor command, light intensityunsupervisedReal Robot reachabilityAug 2018
How iCub Learns to Imitate Use of a Tool Quickly by Recycling the Past Knowledge Learnt During DrawingBhat & Mohan2015observerextroceptiveyesmesoatomicyesyesyesextrinsicDemonstrationregressionSingle-/Multi-step predictiononlinecontinuousyesbidirectionalbothmathematicalPMP, Motor imagerytrajectory, motion shapeself-supervisedReal Robot reachingAug 2018
Recognizing actions with the associative self-organizing mapBuonamente et al.2013observerextroceptivenoglobalatomicyesnoyesnot specifiedGround truthclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalASOMposture vectors from movieunsupervisedBenchmarkIXMAS Aug 2018
Learning actions from human-robot dialoguesCantrell et al.2011agentextroceptivenoglobalatomicnoyesyesnot specifiedHard codedclassificationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalDependency Parsing, NNlanguageunsupervisedReal Robot followingAug 2018
An Efficient Approach for Multi-view Human Action Recognition Based on Bag-of-Key-PosesChaaraoui et al.2012observerextroceptivenoglobalatomicyesnoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalk-Means, NNcenter of mass, euclidean distance, bag of key posessupervisedBenchmarkMuHAVi Aug 2018
Towards a Conceptual Representation of ActionsChella et al.2000agentbothnoglobalatomicyesyesyesnot specifiedGround truthinferencePlanningofflinecategoricalyesunidirectionalbodymathematicalCS, RNNlocation, posessupervisedReal Robot navigationAug 2018
ReadingAct RGB-D action dataset and human action recognition from local featuresChen et al.2014observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalInterest Point Detection, DTW, SVM2D depth and intensity imagessupervisedBenchmarkCAD-60, RGBD-HUDAACT, Chen_2014 Aug 2018
Learning of composite actions and visual categories via grounded linguistic instructions: Humanoid robot simulationsChuang et al.2012agentextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationregressionRecognitionofflinecontinuousyesunidirectionalbodybiomimeticANNlanguage, RGB imagessupervisedSimulation open, close, lift, up, down, move left, rightAug 2018
Platas—Integrating Planning and the Action Language GologClaßen et al.2012observerextroceptivenoglobalatomicnoyesnot specifiednot specifiedLanguageoptimizationPlanningnot specifiedcategoricalnobidirectionalbithmathematicalNon- and Deterministic Planning Languagelanguagenot specifiedSimulation delivering lettersAug 2018
What Will You Do Next? A Cognitive Model for Understanding Others’ Intentions Based on Shared RepresentationsDindo & Chella2013observerbothnoglobalcompoundyesyesyesnot specifiedGround truthclassificationSingle-/Multi-step predictionofflinecategoricalyesunidirectionalbothmathematicalDBNnot specifiednot specifiednot specified  Aug 2018
Hankelet-based action classification for motor intention recognitionDindo et al.2017observerextroceptivenomesoatomicnonoyesnot specifiedGround truthclassificationSingle-/Multi-step predictionofflinecontinuousnounidirectionalbodymathematicalSVMIMU readingssupervisedReal Robot walkingAug 2018
Learn to wipe: A case study of structural bootstrapping from sensorimotor experienceDo et al.2014agentbothnonot specifiedatomicnoyesyesnot specifiedDemonstrationregressionEffect predictiononlinecontinuousyesunidirectionalenvironmentmathematicalDMP, SVRend effector forces, visionsupervisedReal Robot wipingAug 2018
Learning programs is better than learning dynamics: A programmable neural network hierarchical architecture in a multi-task scenarioDonnarumma et al.2015agentproprioceptivenoglobalatomicyesyesyesextrinsicGround truthoptimizationSingle-/Multi-step predictionofflinecontinuousnounidirectionalbodymathematicalHPNNAsonar readingssupervisedSimulation reachingAug 2018
Learning a repertoire of actions with deep neural networksDroniou et al.2014limbbothnoglobalatomicnonoyesnot specifiedDemonstrationregressionSingle-/Multi-step predictionofflinecontinuousyesunidirectionalbothmathematicalANNcartesian positions and velocitiesself-supervisedReal Robot handwritingAug 2018
Bayesian Approaches for Learning of Primitive-Based Compact Representations of Complex Human ActivitiesEndres et al.2016observerextroceptivenoglobalatomicyesyesyesnot specifiedGround truthregressionRecognitionofflinecategoricalnonot specifiedbodymathematicalPCA, ICA, Bayesian BinningEMG dataunsupervisedBenchmark TaekWonDo kicks and movesAug 2018
Probabilistic model-based imitation learningEnglert et al.2013agentproprioceptivenomesoatomicnonoyesnot specifiedDemonstrationoptimizationSingle-/Multi-step predictionofflinecontinuousnounidirectionalbothmathematicalGPjoint configurationssupervisedReal Robot pendulum swing, ball hittingAug 2018
Learning to Recognize Activities from the Wrong View PointFarhadi & Tabrizi2008observerextroceptivenoglobalatomicyesnoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedenvironmentmathematicalNN Classifier, MaxMargin Clustering, PCAHOS, optical flowsupervisedBenchmarkIXMAS Aug 2018
Action Recognition Using Motion Primitives and Probabilistic Edit DistanceFihl et al.2006observerextroceptivenoglobalatomicyesnoyesnot specifiedGround truthinferenceRecognitiononlinecategoricalnonot specifiedbodymathematicalPCA, Probabilistic Edit DistanceSTV, 3D joint configurations, mahalanobis distancesupervisedVirtual Reality arm movementsAug 2018
Incremental action recognition and generalizing motion generation based on goal-directed featuresGräve & Behnke2012observerextroceptivenomesoatomicnonoyesnot specifiedDemonstrationclassificationRecognitionofflinecontinuousnounidirectionalbothmathematicalHMM, GPRobject and hand posessupervisedBenchmark grasping, pushingAug 2018
Synaptic plasticity in a recurrent neural network for versatile and adaptive behaviors of a walking robotGrinke et. al.2015agentextroceptivenoglobalatomicyesnoyesextrinsicExplorationregressionPlanningonlinecontinuousnounidirectionalbodybiomimeticSynaptic scalingterrain information, obstacle heightunsupervisedReal Robot walking, avoid/climbing obstacle, escape from corner/narrow passageAug 2018
Matching Trajectories of Anatomical Landmarks Under Viewpoint, Anthropometric and Temporal TransformsGritai et al.2009observerextroceptivenolocalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalTrajectory Matching 2D motions of landmarkssupervisedBenchmark walking waving, bendingAug 2018
Minimalist plans for interpreting manipulation actionsGuha et al.2013observerbothnomesoatomicnoyesyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalTracking, Planninggripper poses, grasp typesupervisedBenchmarkUMD Aug 2018
Conceptual imitation learning based on functional effects of actionHajimirsadeghi2011agentextroceptivenomesoatomicnonoyesextrinsicDemonstrationregressionSingle-/Multi-step predictiononlinecategoricalyesbidirectionalenvironmentbiomimeticHMMmotion sequencessupervisedReal Robot hand gesturesAug 2018
Realtime manipulation planning system integrating symbolic and geometric planning under interactive dynamics siumlatorHaneda et al.2008agentbothnomesoatomicyesyesyesextrinsicExplorationoptimizationPlanningonlinebothyesbidirectionalbothmathematicalSymbolic and Geometric Planning, Motor imageryobject and robot posessupervisedSimulation stackingAug 2018
Tracking in Action SpaceHerzog & Krüger2012limbextroceptivenomesoatomicyesnoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalPHMM, BPedge featuressupervisedBenchmark reaching, pointing, pushing, graspingAug 2018
Coupled learning of action parameters and forward models for manipulationHöfer & Brock2016agentproprioceptivenoglobalatomicyesyesyesnot specifiedExplorationoptimizationPlanningofflinecategoricalyesunidirectionalenvironmentmathematicalIterative clusteringrelational scene descriptionunsupervisedSimulation pushing, pullingAug 2018
Learning Causality and Intentional ActionsHongeng & Wyatt2008observerextroceptiveyesmesoatomicyesyesyesnot specifiedDemonstrationinferenceRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalBNobject and gripper state, gripper-obj. and obj.-obj. relationsunsupervisedBenchmark reachingAug 2018
Computational modeling of observational learning inspired by the cortical underpinnings of human primatesHourdakis & Trahanias2012observerextroceptivenomesoatomicnonoyesextrinsicDemonstrationoptimizationSingle-/Multi-step predictiononlinecontinuousnounidirectionalnot specifiedbiomimeticLSM, SOM, ANNjoint confiugurations, trajectoriessupervisedSimulation reachingAug 2018
Seamless Integration and Coordination of Cognitive Skills in Humanoid Robots: A Deep Learning ApproachHwang & Tani2017limbextroceptiveyesmesoatomicyesnoyesnot specifiedDemonstrationregressionSingle-/Multi-step predictionofflinecontinuousyesunidirectionalenvironmentbiomimeticVisuo-Motor Deep Dynamic Neural NetworkRGB imagessupervisedReal Robot reach and graspAug 2018
Classification of human actions using pose-based features and stacked auto encoderIjjina & Mohan C2016observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalStacked AEpose datasupervisedBenchmarkCMU MoCap, Berkeley-MHAD Aug 2018
Robust Feature Extraction for Shift and Direction Invariant Action RecognitionJeon et al.2015observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalSVMoptical flowsupervisedBenchmarkKTH Aug 2018
View-Invariant Human Action Recognition Using Exemplar-Based Hidden Markov ModelsJi & Lui2009observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalHMM, EM, k-Means Clustering3D-key poses, human silhouette, optical flowsupervisedBenchmarkIXMAS Aug 2018
A New Framework for View-Invariant Human Action RecognitionJi et al.2010observerextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationclassificationRecognitionofflinecontinuousnounidirectionalbodymathematicalHMMbody key poses, contour shape featuresunsupervisedBenchmarkIXMAS Aug 2018
Study of Human Action Recognition Based on Improved Spatio-temporal FeaturesJi et al.2014observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalSVM3D SIFT, PDIsupervisedBenchmarkKTH Aug 2018
Finding Actions Using Shape FlowsJiang & Martin2008observerextroceptivenoglobalatomicyesnoyesnot specifiedHard codedclassificationRecognitionofflinecategoricalnonot specifiedenvironmentmathematicalTemplate Matchingflow linessupervisedBenchmarkWeizmann Aug 2018
Cross-View Action Recognition from Temporal Self-similaritiesJunejo et al.2015observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalNNC, SVMEDM, SSMsupervisedBenchmarkCMU MoCap, Weizmann, IXMAS Aug 2018
Transfer of Elementary Skills via Human-Robot InteractionKaiser1997agentextroceptiveyesmesoatomicnonoyesnot specifiedDemonstrationoptimizationSingle-/Multi-step predictionofflinecontinuousnounidirectionalenvironmentmathematicalRBF networknot specifiedsupervisedReal Robot force control, peg-in-hole, dockingAug 2018
Improved GLOH Approach for One-Shot Learning Human Gesture RecognitionKarn & Jiang2016observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalkNN, k-Means Clustering, LDBOF, visual codebook, GLOHsupervisedBenchmarkChaLearn Gesture Aug 2018
Natural Language Communication Between Human and Artificial AgentsKemke2006agentextroceptivenomesoatomicnoyesyesnot specifiedHard codedclassificationLanguageofflinecategoricalnounidirectionalbothmathematicalNLP, Ontologiesposes, action verbsupervisedSimulation reaching, grasping, drivingAug 2018
Visual object-action recognition: Inferring object affordances from human demonstrationKjellström et.al.2011limbextroceptivenomesoatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalSVM, CRFHOG, hand posesupervisedBenchmark pour, hammer, openAug 2018
Tensor Representations via Kernel Linearization for Action Recognition from 3D SkeletonsKoniusz et al.2016observerextroceptivenoglobalatomicyesnoyesnot specifiedGround truthclassificationRecognitionofflinecontinuousnonot specifiedbodymathematicalSVM, RBFjoint configurationssupervisedBenchmarkUTKinect Action, Florence3D Action, MSR-Action 3D Aug 2018
Recognizing Action Primitives in Complex Actions Using Hidden Markov ModelsKrüger2006observerextroceptivenoglobalatomicyesyesyesnot specifiedDemonstrationregressionRecognitionofflinecategoricalnounidirectionalbodymathematicalHMMjoint configurationsunsupervisedBenchmarkMoPrim Aug 2018
Using Hidden Markov Models for Recognizing Action Primitives in Complex ActionsKrüger & Grest2007observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalHMMposessupervisedBenchmarkMoPrim Aug 2018
Tracking in object action spaceKrüger & Herzog 2013observerextroceptivenomesoatomicnoyesyesnot specifiedDemonstrationoptimizationRecognitionofflinecontinuousnounidirectionalbothmathematicalPHMM, Bayesbody-fix pointsunsupervisedBenchmark pointing, pouringAug 2018
Object–Action Complexes: Grounded abstractions of sensory–motor processesKrüger et al.2011agentextroceptivenomesobothnoyesyesnot specifiedcombinationregressionPlanningofflinecontinuousyesunidirectionalenvironmentmathematicalNN, KDE, Samplingco-planar contours, location, ECVsupervisedReal Robot grasping, pushingAug 2018
Learning Actions from ObservationsKruger; D. L. Herzog; S. Baby; A. Ude; D. Kragic2010observerextroceptiveyesmesocompoundnoyesyesextrinsicDemonstrationclassificationSingle-/Multi-step predictiononlinecategoricalyesbidirectionalbothmathematicalPHMMmotion sequences, object poseunsupervisedReal Robot move, push, rotate with armAug 2018
An Unsupervised Framework for Action Recognition Using ActemesKulkarni et al.2011observerextroceptivenoglobalcompoundyesyesyesextrinsicLanguageregressionSingle-/Multi-step predictiononlinecategoricalnounidirectionalenvironmentmathematicalk-Means, one-pass DP decoding, HMM3D Visual HullunsupervisedBenchmarkIXMAS Aug 2018
Action representation for planning using truth maintenance systemKulkarni; N. Parameswaran; R. Nagarajan1989agentproprioceptivenomesoatomicyesyesnoextrinsicHard codedinferencePlanningofflinecategoricalnobidirectionalenvironmentmathematicalTruth maintainance system action symbols, statessupervisedSimulation open, stackAug 2018
New approach for action recognition using motion based featuresKumar; P. Sivaprakash2013observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalkNNkey pointssupervisedBenchmarkKTH Aug 2018
Embodiment independent manipulation through action abstractionLaaksonen; J. Felip; A. Morales; V. Kyrki2010limbextroceptivenomesocompoundnoyesyesextrinsicHard codednot specifiednot specifiedofflinenot specifiednot specifiednot specifiednot specifiedmathematicalTransfer Learningpredefined actionssupervisedReal Robot grasp, liftAug 2018
Linking language with embodied and teleological representations of action for humanoid cognitionLallee et al.2010limbextroceptiveyesglobalatomicnonoyesnot specifiedDemonstrationinferenceEffect predictiononlinecategoricalyesbidirectionalenvironmentbiomimeticSpikenet, Temporal segmentation, Rule enginevisibility, moving, contact, languagesupervisedReal Robot cover, uncoverAug 2018
Real-Time Biologically Inspired Action Recognition from Key Poses Using a Neuromorphic ArchitectureLayher et al.2017observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalnot specifiedmathematicalDCNNkey posessupervisedBenchmark bend, jack, jump, run, walk, skip, wave, pick up, sit , rope, push, raise, press, lunge, twist, stretch, touchAug 2018
Learning convolutional action primitives for fine-grained action recognitionLea et al.2016agentbothnomesobothnoyesyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalLatent Convolutional Skip Chain CRFdepth, positions, velocitiessupervisedBenchmark50 Salads, JIGSAWS Aug 2018
Robot skill discovery bases on observed dataLee; J. Chen1996agentextroceptivenoglobalatomicnonoyesextrinsicDemonstrationoptimizationPlanningofflinenot specifiedyesnot specifiedbodymathematicalMultiresolution Global Comp. and Local Coop. Algorithm FSTMunsupervisedSimulation navigationAug 2018
Salient pairwise spatio-temporal interest points for real-time activity recognitionLiu et al.2016observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthinferenceRecognitionofflinecategoricalnounidirectionalbodymathematicalDirected GraphsTSP, SSPsupervisedBenchmarkKTH, ADLs, UT-Interaction Aug 2018
Action recognition using dynamics featuresMansur; Makihara and Yagi2011observerextroceptivenolocalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalHMMtorquesupervisedBenchmark walk, run, march, sit,kump forward, jump in place, hopAug 2018
Semi-supervised Learning of Action Ontology from Domain-Specific CorporaMarkievicz et al.2013observerextroceptivenomesoatomicyesyesyesnot specifiedLanguageclassificationRecognitionofflinecategoricalnounidirectionalbothmathematicalNLP, Ontologies, WSMtextsemi-supervisedBenchmarkCHEMLAB corpus Aug 2018
Grounding Action Words in the Sensorimotor Interaction with the World: Experiments with a Simulated iCub Humanoid RobotMarocco et al.2010agentbothnomesoatomicnonoyesnot specifiedGround truthregressionSingle-/Multi-step predictionofflinecontinuousnounidirectionalenvironmentmathematicalRNNjoint configurations, tactile readings, language, roundnesssupervisedReal Robot crawlingAug 2018
Modelling the Cortical Columnar Organisation for Topological State-Space Representation, and Action PlanningMartinet et al.2008agentproprioceptivenolocalatomicyesnot specifiedyesextrinsicExplorationoptimizationPlanningofflinecontinuousyesbidirectionalbodybiomimeticActivation-Diffusion planninglocation and orientation HPunsupervisedSimulation navigationAug 2018
Extending sensorimotor contingency theory: prediction, planning and action generationMaye & Engl2013agentproprioceptivenoglobalatomicyesyesyesextrinsicExplorationregressionSingle-/Multi-step predictiononlinecontinuousyesunidirectionalbodymathematicalHMMmotor readingsself-supervisedReal Robot navigationAug 2018
Action recognition with appearance–motion features and fast search treesMikolajczyk & Uemura2011observerextroceptiveyesglobalatomicnoyesyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalKD-TreeGLOH, Hessian/Harris-Laplace, MSERsupervisedBenchmarkKTH, Weizmann Aug 2018
Inference Through Embodied Simulation in Cognitive RobotsMohan et al.2013agentbothnomesoatomicyesyesyesextrinsicExplorationregressionSingle-/Multi-step predictiononlinecontinuousyesbidirectionalenvironmentbiomimeticGrowing SOM, PMPsize, color, shape, object wordunsupervisedReal Robot pushingAug 2018
Teaching Humanoids to Imitate ‘Shapes’ of MovementsMohan et al.2010limbextroceptivenolocalatomicnonoyesextrinsicDemonstrationregressionEffect predictionofflinecontinuousnounidirectionalbodymathematicalDirect Linear Transfromlines, ciritical pointssupervisedReal Robot drawingAug 2018
Autonomous Learning of High-Level States and Actions in Continuous EnvironmentsMugan & Kuipers2012agentbothnoglobalatomicyesyesyesintrinsicExplorationoptimizationSingle-/Multi-step predictiononlinecontinuousyesunidirectionalbothmathematicalRL, DBNobject poses and velocitiesunsupervisedSimulation grasping, pushingAug 2018
Adaptive synthesis of dynamically feasible full-body movements for the humanoid robot HRP-2 by flexible combination of learned dynamic movement primitivesMukovskiy et al.2017agentextroceptivenoglobalatomicnoyesyesnot specifiedDemonstrationregressionPlanningofflinecontinuousnounidirectionalnot specifiedmathematicalDMPs, Anechoic Mixing Modeljoint angle trajectoriessupervisedReal Robot walking-reachingAug 2018
A biomimetic approach to robot table tennisMülling et al.2011agentextroceptivenomesoatomicnoyesyesnot specifiedHard codedoptimizationSingle-/Multi-step predictionnot specifiedcontinuousnounidirectionalenvironmentbiomimeticFSMpoints in 3Dnot specifiedReal Robot tennis strokesAug 2018
Acquisition of viewpoint representation in imitative learning from own sensory-motor experiencesNakajo et al.2015agentbothnomesoatomicnoyesyesnot specifiedGround truthregressionSingle-/Multi-step predictionofflinecontinuousyesunidirectionalbothmathematicalCTRNNjoint configurations, RGB imagessupervisedReal Robot touching, reachingAug 2018
Graphical framework for action recognition using temporally dense STIPsNatarajan; P. Banerjee; F. M. Khan; R. Nevatia2009observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalCRFstemporal dense-STIPsupervisedBenchmarkKTH Aug 2018
Development process of functional hierarchy for actions and motor imageryNishimoto & Tani2009agentbothnomesoatomicyesyesyesextrinsicDemonstrationregressionSingle-/Multi-step predictionofflinebothyesbidirectionalbothmathematicalMTRNNjoint configurations, camera angle, object posesupervisedReal Robot push, pull, touchAug 2018
Learning Multiple Goal-Directed Actions Through Self-Organization of a Dynamic Neural Network Model: A Humanoid Robot ExperimentNishimoto et al.2008agentproprioceptivenomesoatomicnonoyesnot specifiedGround truthregressionSingle-/Multi-step predictionofflinecontinuousnounidirectionalenvironmentbiomimeticCTRNNjoint configurationssupervisedReal Robot lift, moveAug 2018
A generative model for developmental understanding of visuomotor experienceNoda; K. Kawamoto; T. Hasuo; K. Sabe2011limbextroceptiveyesglobalatomicnononoextrinsicExplorationclassificationEffect predictiononlinecontinuousnounidirectionalbothbiomimeticHMMappearance in the vision, motor of arm and camera unsupervisedSimulation reaching, interactingAug 2018
Acquiring hand-action models by attention point analysisOgawara et al.2001limbextroceptiveyesmesoatomicnonoyesnot specifiedDemonstrationregressionSingle-/Multi-step predictionofflinecategoricalyesunidirectionalenvironmentmathematicalAttention Point Analysisobject color and shape, timeunsupervisedReal Robot grasp, pick, pour, handoverAug 2018
Hierarchies for Embodied Action PerceptionOgnibene2013observerextroceptivenomesoatomicyesyesyesextrinsiccombinationinferenceSingle-/Multi-step predictionofflinebothyesbidirectionalbothmathematicalBNobject state, gripper state, motor commandsunsupervisedReal Robot manipulationAug 2018
Learning Epistemic Actions in Model-Free Memory-Free Reinforcement Learning: Experiments with a Neuro-robotic ModelOgnibene et al.2013agentextroceptiveyeslocalatomicyesyesyesnot specifiedExplorationnot specifiedEffect predictiononlinenot specifiedyesunidirectionalenvironmentbiomimeticRL, NNeye posture map, arm posture mapsemi-supervisedSimulation reachingAug 2018
A Spiking Neural Network Model of Multi-modal Language Processing of Robot InstructionsPanchev2005agentbothyesglobalatomicyesyesyesextrinsiccombinationregressionSingle-/Multi-step predictionofflinecontinuousyesunidirectionalbodybiomimeticSpiking Neural Networkobject features, frequency map, tactile readingsunsupervisedSimulation navigation, manipulationAug 2018
Comparing Hidden Markov Models and Long Short Term Memory Neural Networks for Learning Action RepresentationsPanzner & Cimiano2016agentextroceptivenoglobalatomicyesyesyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalHMM, LSTMQTCsupervisedSimulation jumps over, jumps on, circling, pushingAug 2018
Self-organizing neural integration of pose-motion features for human action recognitionParisi et al.2015observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodybiomimeticSOM, GWRpose-motion vectorsunsupervisedBenchmark standing, walking, jogging, sitting, lying down, pick up object, jump, fall down, stand upAug 2018
Learning for Goal-directed Actions using RNNPB: Developmental Change of “What to Imitate”Park et al.2017agentproprioceptivenoglobalatomicnoyesyesnot specifiedDemonstrationregressionSingle-/Multi-step predictionofflinecontinuousyesunidirectionalbodymathematicalRNNPB (parametric bias)motor commandssupervisedCombination reachingAug 2018
Learning object, grasping and manipulation activities using hierarchical HMMsPatel et al.2013observerextroceptivenomesocompoundyesyesyesnot specifiedGround truthclassificationSingle-/Multi-step predictionofflinecategoricalnonot specifiedbodymathematicalHHMM, EMhand and object motion/orientation, object classunsupervisedBenchmark reach, grasp, lift, placeAug 2018
Do what i want, not what i did: Imitation of skills by planning sequences of actionsPaxton et al.2016agentbothnomesoatomicyesyesyesnot specifiedGround truthinferencePlanningofflinecategoricalnounidirectionalenvironmentmathematicalGMMtime, gripper commands, object transformssupervisedReal Robot structure buildingAug 2018
Joint movement similarities for robust 3D action recognition using skeletal dataPazhoummand-Dar et al.2015observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalLCSS3D body skeletonsupervisedBenchmarkMSR-Action 3D, HDM05 Aug 2018
What should I do next? Using shared representations to solve interaction problemsPezzulo & Dindo2011observerbothnoglobalatomicyesyesyesnot specifiedDemonstrationinferenceSingle-/Multi-step predictiononlinecontinuousyesbidirectionalbothmathematicalDBNmovement kinematicsunsupervisedVirtual Reality reach, put, turnAug 2018
Clustering of human actions using invariant body shape descriptor and dynamic time warpingPierobon; M. Marcon; A. Sarti; S. Tubaro2005observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalDTW3D shape descriptorssupervisedBenchmark pointing, crouching down, kickAug 2018
Audio-visual classification and detection of human manipulation actionsPieropan et al.2014observerextroceptivenomesoatomicnoyesyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalHMMposes, distancessupervisedBenchmark open, pour, closeAug 2018
Cross-modal and scale-free action representations through enactionPitti et al.2009agentbothyesmesoatomicnonoyesnot specifiedDemonstrationclassificationRecognitionofflinecontinuousnounidirectionalenvironmentbiomimeticRNN, STDPsaliency maps, touchself-supervisedSimulation graspingAug 2018
Fast action recognition using negative space featuresRahman et al.2014observerextroceptiveyesglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalNNnegative space action descriptorssupervisedBenchmarkKTH, Weizmann, Fish action Aug 2018
Transferring skills to humanoid robots by extracting semantic representations from observations of human activitiesRamirez-Amaro et al.2017limbextroceptivenomesobothnoyesyesextrinsicDemonstrationinferencePlanningofflinecategoricalnounidirectionalnot specifiedmathematicalDecision TreecolorsupervisedReal Robot making a pancakeAug 2018
A new invariant descriptor for action recognition based on spherical harmonicsRazzaghi et al.2012observerextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalSVM, spherical harmonicsspatio-temporal volumesupervisedBenchmarkKTH, Weizmann, IXMAS, Robust Aug 2018
View-independent human action recognition with Volume Motion Template on single stereo cameraRoh et al.2010observerextroceptivenoglobalatomicnonoyesnot specifiedDemonstrationinferenceRecognitionofflinecontinuousnounidirectionalenvironmentmathematicalVMT, PMTsilhouette and depth mapsunsupervisedBenchmark hand movements, bowingAug 2018
A framework for heading-guided recognition of human activityRosales & Sclaroff2003observerextroceptiveyesglobalatomicnonoyesnot specifiedDemonstrationoptimizationRecognitionofflinecontinuousnounidirectionalbodymathematicalEKF, PCA, EM3D trajectoriesunsupervisedBenchmark walking, running, r.blading, bikingAug 2018
Hand-Object Interaction and Precise Localization in Transitive Action RecognitionRosenfeld & Ullman2016observerextroceptiveyeslocalatomicnonononot specifiedGround truthregressionRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalDAG-RNN, SVMface, hand object location probability mapsupervisedBenchmarkStanford-40 Actions Aug 2018
Human activity recognition in videos using a single exampleRoshtkhari & Levine2013observerextroceptiveyesglobalatomicnonoyesnot specifiedGround truthregressionRecognitionofflinecategoricalnounidirectionalbodymathematicalSimilarity mappingHOG, SIFTsupervisedBenchmarkKTH, Weizmann, MSR II Aug 2018
A Multi-Scale Hierarchical Codebook Method for Human Action Recognition in Videos Using a Single ExampleRoshtkhari; M. D. Levine2012observerextroceptivenoglobalatomicnonoyesintrinsicExplorationclassificationRecognitiononlinenot specifiednot specifiednot specifiednot specifiedmathematicalBag of video words, Code bookSTVsupervisedBenchmarkKTH, Weizmann, MSR II Aug 2018
Learning the Consequences of Actions: Representing Effects as Feature ChangesRudolph et al.2010agentextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationinferenceSingle-/Multi-step predictionofflinecontinuousyesunidirectionalenvironmentmathematicalBNobject pose and presencesupervisedSimulation can tossing, ball throwingAug 2018
Learning sequential and continuous controlRyan & Andreae1993agentextroceptivenoglobalatomicyesyesyesnot specifiedExplorationregressionSingle-/Multi-step predictiononlinecontinuousnounidirectionalbodybiomimeticPP, CMACpositionunsupervisedSimulation reachabilityAug 2018
Action Recognition Robust to Background Clutter by Using Stereo VisionSanchez-Riera et al.2012observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalk-Means, SVMBOW, Scene/Optical FlowsupervisedBenchmarkRavel Aug 2018
Primitive Based Action Representation and RecognitionSanmohan & Krüger2009observerextroceptivenoglobalatomicnoyesyesnot specifiedDemonstrationinferenceRecognitionofflinecategoricalnounidirectionalbodymathematicalHMM, SCFGtrajectoriesunsupervisedCombination walking, grasping, pushingAug 2018
Learning Discriminative Space–Time Action Parts from Weakly Labelled VideosSapienza et al.2014observerextroceptivenomesocompoundyesyesyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalSVM-mapBOFsemi-supervisedBenchmarkKTH, YouTube, Hollywood2, HMDB Aug 2018
Encoding human actions with a frequency domain approachShah et al.2016observerproprioceptivenomesobothyesnoyesnot specifiedDemonstrationclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalFFT, PCA, Clusteringjoint configurationssemi-supervisedBenchmarkHDM05 Aug 2018
Learning Skeleton Stream Patterns with Slow Feature Analysis for Action RecognitionShan et al.2014observerextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalSFA, k-Meansskeletal joint streamsunsupervisedBenchmarkMSR-Action 3D Aug 2018
Teaching Robots New Actions through Natural Language InstructionsShe et al.2014agentbothnomesoatomicnoyesyesextrinsicGround truthinferencePlanningonlinecategoricalyesunidirectionalbothmathematicalNLP, Vision Graphslanguage, object poses, distancesself-supervisedReal Robot reaching, graspingAug 2018
Integration of spatial and temporal contexts for action recognition by self organizing neural networksShimozaki; Y. Kuniyoshi2003observerextroceptiveyesglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedbiomimeticSOM2D animationsupervisedBenchmark grasp, carry, placeAug 2018
Navigating mobile robots with a modular neural architectureSilva & Ribeiro2003agentproprioceptivenomesoatomicyesnoyesextrinsicDemonstrationregressionSingle-/Multi-step predictionofflinecategoricalyesunidirectionalbodymathematicalMNNrobot state, world statesupervisedReal Robot navigationAug 2018
Spatiotemporal representation of 3D skeleton joints-based action recognition using modified spherical harmonicsSlaih and Chahir2016observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecontinuousnounidirectionalbodymathematicalSpherical Harmonics, ELM3D body skeletonsupervisedBenchmarkMSR-Action 3D, G3D, Florence3D Action, UTKinect-Action Aug 2018
Leaving Some Stones Unturned: Dynamic Feature Prioritization for Activity Detection in Streaming VideoSu & Grauman2016observerextroceptivenomesocompoundyesyesyesnot specifiedGround truthregressionSingle-/Multi-step predictionofflinebothnonot specifiedenvironmentmathematicalMDP, GMMBOO, CNN-featuressupervisedBenchmarkADLs, UCF-101 Aug 2018
A sub-symbolic process underlying the usage-based acquisition of a compositional representation: Results of robotic learning experiments of goal-directed actionsSugita; J. Tani2008agentextroceptivenolocalatomicnoyesyesextrinsicDemonstrationregressionSingle-/Multi-step predictionofflinecategoricalyesbidirectionalbodybiomimeticNNcolored patches, speed of wheelsupervisedSimulation reachingAug 2018
Action Disambiguation Analysis Using Normalized Google-Like Distance CorrelogramSun & Liu2013observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalNGLD+BOW, kNN, SVMSTIP, 3D-Sift, Spatial-Temporal Interest PointsupervisedBenchmarkWeizmann, UCF Sports Aug 2018
A novel hierarchical Bag-of-Words model for compact action representationSun et al.2016observerextroceptiveyesglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalHierarchical BOW, SVM2D imagessupervisedBenchmarkHollywood2, Olympic Sports, YouTube, HMDB Aug 2018
A unified representation for reasoning about robot actions, processes, and their effects on objectsTenorth & Beetz2012agentbothyesmesoatomicyesyesyesnot specifiedHard codedinferencePlanningofflinebothnounidirectionalbothmathematicalOntologies, Logic Programmingobject shape and pose, body posessupervisedReal Robot making pancakeAug 2018
From motor to sensory processing in mirror neuron computational modellingTessitore et al.2010agentbothnomesocompoundyesyesyesnot specifiedGround truthclassificationSingle-/Multi-step predictionofflinecontinuousyesbidirectionalbodybiomimeticPCA, NN, MDNscene descriptorssupervisedBenchmarkHuman-Grasp Aug 2018
Ubiquitous robotics in physical human action recognition: A comparison between dynamic ANNs and GPTheodoridis et al.2008observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalANN, Genetic Programmingkinematic body modelsupervisedBenchmark daily activitiesAug 2018
Behavior Histograms for Action Recognition and Human DetectionThurau2007observerextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalk-MeansHOGunsupervisedBenchmarkWeizmann Aug 2018
n-Grams of Action Primitives for Recognizing Human BehaviorThurau & Hlavac2007observerextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalPCA, NGrams, Ward's clusteringeigenshapesunsupervisedBenchmarkWeizmann Aug 2018
Recognizing Human Actions by Their PoseThurau & Hlavac2009observerextroceptivenoglobalatomicyesyesyesnot specifiedGround truthclassificationSingle-/Multi-step predictionofflinecategoricalnonot specifiedbodymathematicalk-Means Clustering, NMFprototypical poses, HOGsemi-supervisedBenchmarkWeizmann Aug 2018
Joint classification of actions and object state changes with a latent variable discriminative modelVafeias & Ramamoorthy2014observerextroceptivenomesoatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalCRFobject pose, body posesupervisedBenchmark drink, push, stack, readAug 2018
Rational imitation for robots: the cost difference modelVanderlest & Winfield2017agentextroceptivenoglobalatomicnonoyesnot specifiedDemonstrationoptimizationPlanningofflinecontinuousnounidirectionalbodymathematicalCost Difference Modelnot specifiedunsupervisedReal Robot navigationAug 2018
On the improvement of human action recognition from depth map sequences using Space–Time Occupancy PatternsVieira et al.2013observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalAction graphs3D body skeletonsupervisedBenchmarkMSR-Action 3D Aug 2018
STOP: Space-Time Occupancy Patterns for 3D Action Recognition from Depth Map SequencesVieira et al.2012observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationEffect predictionofflinecategoricalnonot specifiedbodymathematicalSVM, PCAPCA-STOPsupervisedBenchmarkMSR-Action 3D Aug 2018
Multiple Kernel Learning and Optical Flow for Action Recognition in RGB-D VideoViet et al.2015observerextroceptivenoglobalatomicyesnononot specifiedDemonstrationclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalMKL classificationdense optical flow, SPHOF on RGB-DsupervisedBenchmarkMSR-Daily Activity, 3D ActionPairs Aug 2018
Probabilistic semantic models for manipulation action representation and extractionVuga et al.2014agentextroceptivenomesoatomicnoyesyesnot specifiedDemonstrationoptimizationRecognitionofflinecontinuousnounidirectionalenvironmentmathematicalSECcolor, surfacesupervisedBenchmark pouring, openingAug 2018
Power difference template for action recognitionWang et al.2017observerextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalBOW, SVMnormalized projection histogram, motion kinetic velocitysupervisedBenchmarkKTH, UCF-Sports, UCF-101, HMDB Aug 2018
Hierarchical interpretation of human activities using competitive learningWechsler; Z. Duric; Fayin Li2002observerextroceptivenomesocompoundnonoyesnot specifiedGround truthclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalLVQ, clusteringmotion parameterssemi-supervisedBenchmark striking, grinding, swing, stirring Aug 2018
Unsupervised learning of reflexive and action-based affordances to model adaptive navigational behaviorWeiller et al.2010agentextroceptivenoglobalatomicyesyesyesnot specifiedExplorationinferencePlanningonlinecontinuousyesunidirectionalbodymathematicalPlace Fields, Geometrical transition matrixaffordancesunsupervisedReal Robot avoidance, navigationAug 2018
Grounding Neural Robot Language in ActionWermter et al.2005observerextroceptivenoglobalatomicyesyesyesextrinsiccombinationregressionSingle-/Multi-step predictionofflinebothyesbidirectionalbodybiomimeticHelmholtz machine, SOMpose, motor direction, action wordunsupervisedSimulation go, pick, liftAug 2018
Efficient Action Recognition with MoFREAKWhiten; Laganière and Bilodeau2013observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalSVMMoFREAK descriptorsupervisedBenchmarkKTH, HMDB Aug 2018
A Simple Ontology of Manipulation Actions Based on Hand-Object RelationsWörgötter et al.2013observerextroceptivenomesoatomicnonoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalSEC, Ontologiesobject and hand posessupervisedBenchmark manipulationAug 2018
Integration of Heterogeneity for Human-Friendly Robotic OperationsXi & Tarn1999agentextroceptivenoglobalatomicnoyesyesextrinsicDemonstrationoptimizationSingle-/Multi-step predictiononlinecontinuousyesunidirectionalbodymathematicalMapping functionforce, posessupervisedReal Robot obstacle avoidanceAug 2018
Human action recognition framework by fusing multiple featuresXiao & Cheng2013observerextroceptiveyesglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinenot specifiednot specifiednot specifiednot specifiedmathematicalMMI, PSVMdepth motion maps, visual words, HOGsupervisedBenchmarkMSR-Action 3D Aug 2018
Human action learning via hidden Markov modelYang et al.1997agentextroceptivenomesoatomicnonoyesnot specifiedGround truthinferenceSingle-/Multi-step predictionofflinecontinuousyesunidirectionalenvironmentmathematicalHMMcartesian gripper trajectorysupervisedReal Robot replace taskAug 2018
Manipulation action tree bank: A knowledge resource for humanoidsYang et al.2014not specifiednot specifiednonot specifiedcompoundyesyesyesnot specifiedGround truthclassificationPlanningofflinenot specifiednot specifiednot specifiednot specifiedmathematicalTree banksaccelerometer, RGB-D / RGBsupervisedBenchmark50 Salads, TACoS Aug 2018
One-shot learning based pattern transition map for action early recognitionYi et al.2017observerextroceptivenoglobalbothyesyesyesnot specifiedDemonstrationregressionRecognitionofflinecategoricalnounidirectionalenvironmentmathematicalPattern transition maps, Q-Learningpose datasemi-supervisedBenchmark3D ActionPairs, SYSU 3D HOI Aug 2018
Online Multimodal Ensemble Learning Using Self-Learned Sensorimotor RepresentationsZambelli & Demiris2017agentbothnoglobalatomicyesyesyesnot specifiedExplorationregressionSingle-/Multi-step predictiononlinecontinuousyesunidirectionalbothmathematicalEnsemble Learningjoints, RGB images, touch, soundself-supervisedReal Robot piano playingAug 2018
Learning the spatial semantics of manipulation actions through preposition groundingZampogiannis et al.2015agentextroceptivenomesoatomicyesnononot specifiedGround truthclassificationRecognitionofflinebothyesunidirectionalenvironmentmathematicalCollection of PVSRGB-D video, spatial relation predicates on tracked objectsunsupervisedBenchmark pour, transfer, stack, stirAug 2018
View-Independent Human Action Recognition by Action Hypersphere in Nonlinear SubspaceZhang & Zhuang2007observerextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalHypersphere classificationmotion history image, polar faturesunsupervisedBenchmarkIXMAS Aug 2018
Unified robot learning of action labels and motion trajectories from 3D human skeletal dataZhang et al.2016observerproprioceptivenoglobalatomicnonoyesnot specifiedDemonstrationclassificationSingle-/Multi-step predictionofflinecategoricalyesunidirectionalenvironmentmathematicalDTW, GMM, SVM, GMRbone vectorssemi-supervisedBenchmarkMSR-Daily Activity Aug 2018
Motion Context: A New Representation for Human Action RecognitionZhang et al.2008observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnonot specifiedbodymathematicalSVM, PLSAmotion words, motion index, motion contextsupervisedBenchmarkKTH, Weizmann Aug 2018
kPose: A New Representation For Action RecognitionZhou et al.2010observerextroceptivenoglobalatomicyesnoyesnot specifiedDemonstrationclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalPose Weighted Distribution Modelprototypical poses, HOGunsupervisedBenchmarkWeizmann Aug 2018
Human action recognition using multi-layer codebooks of key poses and atomic motionsZhu et al.2016observerextroceptivenoglobalatomicnonoyesnot specifiedGround truthclassificationRecognitionofflinecategoricalnounidirectionalbodymathematicalCodebooks, SVM, NBNN, RF3D body skeletonsupervisedBenchmarkCAD-60, MSRC-12 Aug 2018
Bootstrapping Q-Learning for Robotics from Neuro-Evolution ResultsZimmer & Doncieux2017agentbothyesglobalatomicyesyesyesintrinsicExplorationoptimizationPlanningofflinecategoricalyesunidirectionalenvironmentmathematicalPolicy Search, Clusteringdistance, tactile, poses, unsupervisedSimulation ball collecting, box pushingAug 2018
Title Author Year Perspective Stimuli Selective Attention Granularity Abstraction Competition Sequencing Generalization Motivation Acquisition Prediction Exploitation Learning Discretization Grounding Associativity Effect Correspondence Formulation Method Features Training Evaluation Datasets Actions Date Added

Please cite as follows:
 @Article{Zech-2019-IJRR,
    title = {{Action representations in robotics: A taxonomy and systematic Classification}},
    author = {Zech, Philipp and Renaudo, Erwan and Haller, Simon and Zhang, Xiang and Piater, Justus},
    journal = {{International Journal of Robotics Research}},
    year = 2019,
    publisher = {SAGE},
    doi = {10.1177/0278364919835020},
    url = {http://dx.doi.org/10.1177/0278364919835020}
  }

Updates:

  • Initial release of this page (08/2018)
  • Update Publication (03/2019)