A survey of vision-based methods for action representation, segmentation and recognition

Authors:
Daniel Weinland;Remi Ronfard;Edmond Boyer
Affiliations:
Deutsche Telekom Laboratories, Berlin Institute of Technology, Berlin, Germany;INRIA - Team Lear Grenoble, France;INRIA - Team Perception Grenoble, France
Venue:
Computer Vision and Image Understanding
Year:
2011

Citing 80
Cited 38

A tutorial on hidden Markov models and selected applications in speech recognition

Readings in speech recognition
Shape and motion from image streams under orthography: a factorization method

International Journal of Computer Vision
Towards model-based recognition of human movements in image sequences

CVGIP: Image Understanding
Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review

IEEE Transactions on Pattern Analysis and Machine Intelligence
View-Invariant Analysis of Cyclic Motion

International Journal of Computer Vision
Parametric Hidden Markov Models for Gesture Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Discovery and Segmentation of Activities in Video

IEEE Transactions on Pattern Analysis and Machine Intelligence
Recognition of Visual Activities and Interactions by Stochastic Parsing

IEEE Transactions on Pattern Analysis and Machine Intelligence
The Recognition of Human Movement Using Temporal Templates

IEEE Transactions on Pattern Analysis and Machine Intelligence
Natural Language Description of Human Activities from Video Images Based on Concept Hierarchy of Actions

International Journal of Computer Vision
View-Invariant Representation and Recognition of Actions

International Journal of Computer Vision
Stochastic Tracking of 3D Human Figures Using 2D Image Motion

ECCV '00 Proceedings of the 6th European Conference on Computer Vision-Part II
On Automatic Selection of Temporal Scales in Time-Causal Scale-Space

AFPAC '97 Proceedings of the International Workshop on Algebraic Frames for the Perception-Action Cycle
Unsupervised Analysis of Human Gestures

PCM '01 Proceedings of the Second IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing
Matching with Shape Contexts

CBAIVL '00 Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'00)
Coupled hidden Markov models for complex action recognition

CVPR '97 Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97)
Learning and Recognizing Human Dynamics in Video Sequences

CVPR '97 Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97)
Action Recognition Using Probabilistic Parsing

CVPR '98 Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Invariant features for 3-D gesture recognition

FG '96 Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96)
View-Based Interpretation of Real-Time Optical Flow for Gesture Recognition

FG '98 Proceedings of the 3rd. International Conference on Face & Gesture Recognition
Recognition of human body motion using phase space constraints

ICCV '95 Proceedings of the Fifth International Conference on Computer Vision
3D Tracking of Human Locomotion: A Tracking as Recognition Approach

ICPR '02 Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 1 - Volume 1
Learning visual behavior for gesture analysis

ISCV '95 Proceedings of the International Symposium on Computer Vision
Real-time American Sign Language recognition from video using hidden Markov models

ISCV '95 Proceedings of the International Symposium on Computer Vision
Shadow Puppetry

ICCV '99 Proceedings of the International Conference on Computer Vision-Volume 2 - Volume 2
Boundaries of Visual Motion

Boundaries of Visual Motion
Finding Periodicity in Space and Time

ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
Parameterized Modeling and Recognition of Activities

ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
Inference of Human Postures by Classification of 3D Human Body Shape

AMFG '03 Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures
Space-time Interest Points

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Recognizing Action at a Distance

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Face recognition: A literature survey

ACM Computing Surveys (CSUR)
Recognition of two-person interactions using a hierarchical Bayesian network

IWVS '03 First ACM SIGMM international workshop on Video surveillance
Distinctive Image Features from Scale-Invariant Keypoints

International Journal of Computer Vision
On the use of Anthropometry in the Invariant Analysis of Human Actions

ICPR '04 Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 2 - Volume 02
Recognizing Human Actions: A Local SVM Approach

ICPR '04 Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 3 - Volume 03
Human Action Segmentation via Controlled Use of Missing Data in HMMs

ICPR '04 Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 4 - Volume 04
Shape representations and algorithms for three-dimensional model retrieval

Shape representations and algorithms for three-dimensional model retrieval
Histograms of Oriented Gradients for Human Detection

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01
Actions Sketch: A Novel Action Representation

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01
Object Recognition with Features Inspired by Visual Cortex

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Combining Image Regions and Human Activity for Indirect Object Recognition in Indoor Wide-Angle Views

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Exploring the Space of a Human Action

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Recognizing Human Actions in Videos Acquired by Uncalibrated Moving Cameras

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Efficient Visual Event Detection Using Volumetric Features

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Detecting Irregularities in Images and in Video

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Behaviour Understanding in Video: A Combined Method

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Actions as Space-Time Shapes

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision - Volume 2
Conditional Random Fields for Contextual Human Motion Recognition

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision - Volume 2
Video-based Human Action Classi.cation with Ambiguous Correspondences

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops - Volume 03
Recovering 3D Human Pose from Monocular Images

IEEE Transactions on Pattern Analysis and Machine Intelligence
The Function Space of an Activity

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1
Hidden Conditional Random Fields for Gesture Recognition

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Automatic Discovery of Action Taxonomies from Multiple Views

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Unsupervised Discovery of Action Classes

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
HMM-based Human Action Recognition Using Multiview Image Sequences

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 01
Volume Motion Template for View-Invariant Gesture Recognition

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 02
A survey of advances in vision-based human motion capture and analysis

Computer Vision and Image Understanding - Special issue on modeling people: Vision-based understanding of a person's shape, appearance, movement, and behaviour
Free viewpoint action recognition using motion history volumes

Computer Vision and Image Understanding - Special issue on modeling people: Vision-based understanding of a person's shape, appearance, movement, and behaviour
A Language for Human Action

Computer
Behavior recognition via sparse spatio-temporal features

ICCCN '05 Proceedings of the 14th International Conference on Computer Communications and Networks
Vision-based hand pose estimation: A review

Computer Vision and Image Understanding
A 3-dimensional sift descriptor and its application to action recognition

Proceedings of the 15th international conference on Multimedia
The Representation and Matching of Pictorial Structures

IEEE Transactions on Computers
View invariant gesture recognition using the CSEM SwissRanger SR-2 camera

International Journal of Intelligent Systems Technologies and Applications
Learning to Recognize Activities from the Wrong View Point

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Scale Invariant Action Recognition Using Compound Features Mined from Dense Spatio-temporal Corners

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Human Activity Recognition with Metric Learning

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Cross-View Action Recognition from Temporal Self-similarities

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part II
Spatial-Temporal correlatons for unsupervised action classification

WMVC '08 Proceedings of the 2008 IEEE Workshop on Motion and video Computing
Histogram of oriented rectangles: A new pose descriptor for human action recognition

Image and Vision Computing
Human motion analysis via statistical motion processing and sequential change detection

Journal on Image and Video Processing - Special issue on video-based modeling, analysis, and recognition of human motion
Semi-latent Dirichlet allocation: a hierarchical model for human action recognition

Proceedings of the 2nd conference on Human motion: understanding, modeling, capture and animation
Human action recognition using distribution of oriented rectangular patches

Proceedings of the 2nd conference on Human motion: understanding, modeling, capture and animation
Detecting unusual activity in video

CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Learning dynamics for exemplar-based gesture recognition

CVPR'03 Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition
Recognition and segmentation of 3-d human action using HMM and multi-class adaboost

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part IV
3D human action recognition using spatio-temporal motion templates

ICCV'05 Proceedings of the 2005 international conference on Computer Vision in Human-Computer Interaction
A survey on visual surveillance of object motion and behaviors

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Quantifying and recognizing human movement patterns from monocular video Images-part I: a new framework for modeling human motion

IEEE Transactions on Circuits and Systems for Video Technology

Automatic human activity recognition in video using background modeling and spatio-temporal template matching based technique

ACAI '11 Proceedings of the International Conference on Advances in Computing and Artificial Intelligence
Learning semantic features for action recognition via diffusion maps

Computer Vision and Image Understanding
Continuous body and hand gesture recognition for natural human-computer interaction

ACM Transactions on Interactive Intelligent Systems (TiiS) - Special Issue on Affective Interaction in Natural Environments
Part-based motion descriptor image for human action recognition

Pattern Recognition
Efficient spatio-temporal edge descriptor

MMM'12 Proceedings of the 18th international conference on Advances in Multimedia Modeling
Human action recognition using a fast learning fully complex-valued classifier

Neurocomputing
Small gestures go a long way: how many bits per gesture do recognizers actually need?

Proceedings of the Designing Interactive Systems Conference
A semantic-based probabilistic approach for real-time video event recognition

Computer Vision and Image Understanding
3D Imaging for hand gesture recognition: Exploring the software-hardware interaction of current technologies

3D Research
Vision based method for object classification and multiple human activity recognition in video survelliance system

Proceedings of the CUBE International Information Technology Conference
Hierarchical and incremental event learning approach based on concept formation models

Neurocomputing
In defence of negative mining for annotating weakly labelled data

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part III
Action recognition using subtensor constraint

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part III
Action recognition robust to background clutter by using stereo vision

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume Part I
Consensus algorithms in a multi-agent framework to solve PTZ camera reconfiguration in UAVs

ICIRA'12 Proceedings of the 5th international conference on Intelligent Robotics and Applications - Volume Part I
On recognizing actions in still images via multiple features

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume Part III
An efficient approach for multi-view human action recognition based on bag-of-key-poses

HBU'12 Proceedings of the Third international conference on Human Behavior Understanding
Evaluating the impact of frame rate on video based human action recognition

Proceedings of the 27th Conference on Image and Vision Computing New Zealand
Boosted key-frame selection and correlated pyramidal motion-feature representation for human action recognition

Pattern Recognition
Human action recognition employing negative space features

Journal of Visual Communication and Image Representation
A survey of video datasets for human action and activity recognition

Computer Vision and Image Understanding
Design of an efficient framework for fast prototyping of customized human-computer interfaces and virtual environments for rehabilitation

Computer Methods and Programs in Biomedicine
Sparse representation for recognizing object-to-object actions under occlusions

Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service
Human action recognition with salient trajectories

Signal Processing
Visual abnormal behavior detection based on trajectory sparse reconstruction analysis

Neurocomputing
Subject independent human action recognition using spatio-depth information and meta-cognitive RBF network

Engineering Applications of Artificial Intelligence
Discriminative two-level feature selection for realistic human action recognition

Journal of Visual Communication and Image Representation
A multi-modal gesture recognition system using audio, video, and skeletal joint data

Proceedings of the 15th ACM on International conference on multimodal interaction
Fast action recognition using negative space features

Expert Systems with Applications: An International Journal
A local descriptor based on Laplacian pyramid coding for action recognition

Pattern Recognition Letters
Kernel analysis on Grassmann manifolds for action recognition

Pattern Recognition Letters
A survey of human motion analysis using depth imagery

Pattern Recognition Letters
Silhouette-based human action recognition using sequences of key poses

Pattern Recognition Letters
Editor's Choice Article: Human activity recognition in videos using a single example

Image and Vision Computing
Octree segmentation based calling gesture recognition for elderly care robot

Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication
Classifying web videos using a global video descriptor

Machine Vision and Applications
On the improvement of human action recognition from depth map sequences using Space-Time Occupancy Patterns

Pattern Recognition Letters
Recognizing activities in multiple views with fusion of frame judgments

Image and Vision Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Action recognition has become a very important topic in computer vision, with many fundamental applications, in robotics, video surveillance, human-computer interaction, and multimedia retrieval among others and a large variety of approaches have been described. The purpose of this survey is to give an overview and categorization of the approaches used. We concentrate on approaches that aim on classification of full-body motions, such as kicking, punching, and waving, and we categorize them according to how they represent the spatial and temporal structure of actions; how they segment actions from an input stream of visual data; and how they learn a view-invariant representation of actions.