Visual object-action recognition: Inferring object affordances from human demonstration

Authors:
Hedvig Kjellström;Javier Romero;Danica Kragić
Affiliations:
Computational Vision and Active Perception Lab, Centre for Autonomous Systems School of Computer Science and Communication, KTH, SE-100 44 Stockholm, Sweden;Computational Vision and Active Perception Lab, Centre for Autonomous Systems School of Computer Science and Communication, KTH, SE-100 44 Stockholm, Sweden;Computational Vision and Active Perception Lab, Centre for Autonomous Systems School of Computer Science and Communication, KTH, SE-100 44 Stockholm, Sweden
Venue:
Computer Vision and Image Understanding
Year:
2011

Citing 22
Cited 6

Recognition by functional parts

Computer Vision and Image Understanding - Special issue of funtion-based vision
Generic object recognition using form and function

Generic object recognition using form and function
Combining labeled and unlabeled data with co-training

COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
An introduction to support Vector Machines: and other kernel-based learning methods

An introduction to support Vector Machines: and other kernel-based learning methods
Contextual Priming for Object Detection

International Journal of Computer Vision
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Task-oriented generation of visual sensing strategies

ICCV '95 Proceedings of the Fifth International Conference on Computer Vision
Fast Pose Estimation with Parameter-Sensitive Hashing

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Histograms of Oriented Gradients for Human Detection

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01
Combining Image Regions and Human Activity for Indirect Object Recognition in Indoor Wide-Angle Views

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Learning Temporal Sequence Model from Partially Labeled Data

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
A survey of advances in vision-based human motion capture and analysis

Computer Vision and Image Understanding - Special issue on modeling people: Vision-based understanding of a person's shape, appearance, movement, and behaviour
Conditional models for contextual human motion recognition

Computer Vision and Image Understanding - Special issue on modeling people: Vision-based understanding of a person's shape, appearance, movement, and behaviour
Robotic Grasping of Novel Objects using Vision

International Journal of Robotics Research
Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Simultaneous Visual Recognition of Manipulation Actions and Manipulated Objects

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part II
Cognitive agents - a procedural perspective relying on the predictability of Object-Action-Complexes (OACs)

Robotics and Autonomous Systems
Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Functional object class detection based on learned affordance cues

ICVS'08 Proceedings of the 6th international conference on Computer vision systems
Learning methods for generic object recognition with invariance to pose and lighting

CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Learning Object Affordances: From Sensory--Motor Coordination to Imitation

IEEE Transactions on Robotics

On the optimization of Hierarchical Temporal Memory

Pattern Recognition Letters
A framework for combined recognition of actions and objects

ICCVG'12 Proceedings of the 2012 international conference on Computer Vision and Graphics
Graspable parts recognition in man-made 3d shapes

ACCV'12 Proceedings of the 11th Asian conference on Computer Vision - Volume Part II
Real web community based automatic image annotation

Computers and Electrical Engineering
Non-parametric hand pose estimation with object context

Image and Vision Computing
Object-object interaction affordance learning

Robotics and Autonomous Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper investigates object categorization according to function, i.e., learning the affordances of objects from human demonstration. Object affordances (functionality) are inferred from observations of humans using the objects in different types of actions. The intended application is learning from demonstration, in which a robot learns to employ objects in household tasks, from observing a human performing the same tasks with the objects. We present a method for categorizing manipulated objects and human manipulation actions in context of each other. The method is able to simultaneously segment and classify human hand actions, and detect and classify the objects involved in the action. This can serve as an initial step in a learning from demonstration method. Experiments show that the contextual information improves the classification of both objects and actions.