3D mean-shift tracking of human body parts and recognition of working actions in an industrial environment

Authors:
Markus Hahn;Fuad Quronfuleh;Christian Wöhler;Franz Kummert
Affiliations:
Daimler AG, Group Research and Advanced Engineering, Ulm, Germany;Daimler AG, Group Research and Advanced Engineering, Ulm, Germany;Image Analysis Group, Dortmund University of Technology, Dortmund, Germany;Applied Informatics, Bielefeld University, Bielefeld, Germany
Venue:
HBU'10 Proceedings of the First international conference on Human behavior understanding
Year:
2010

Citing 8
Cited 1

Mean Shift, Mode Seeking, and Clustering

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Probabilistic Framework for Matching Temporal Trajectories: CONDENSATION-Based Recognition of Gestures and Expressions

ECCV '98 Proceedings of the 5th European Conference on Computer Vision-Volume I - Volume I
Kernel-Based Object Tracking

IEEE Transactions on Pattern Analysis and Machine Intelligence
Real Time Face and Object Tracking as a Component of a Perceptual User Interface

WACV '98 Proceedings of the 4th IEEE Workshop on Applications of Computer Vision (WACV'98)
Efficient Dense Scene Flow from Sparse or Dense Stereo Data

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching

ICVS '09 Proceedings of the 7th International Conference on Computer Vision Systems: Computer Vision Systems
3D action recognition and long-term prediction of human motion

ICVS'08 Proceedings of the 6th international conference on Computer vision systems
An object-oriented approach using a top-down and bottom-up process for manipulative action recognition

DAGM'06 Proceedings of the 28th conference on Pattern Recognition

Challenges of human behavior understanding

HBU'10 Proceedings of the First international conference on Human behavior understanding

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this study we describe a method for 3D trajectory based recognition of and discrimination between different working actions in an industrial environment. A motion-attributed 3D point cloud represents the scene based on images of a small-baseline trinocular camera system. A two-stage mean-shift algorithm is used for detection and 3D tracking of all moving objects in the scene. A sequence of working actions is recognised with a particle filter based matching of a non-stationary Hidden Markov Model, relying on spatial context and a classification of the observed 3D trajectories. The system is able to extract an object performing a known action out of a multitude of tracked objects. The 3D tracking stage is evaluated with respect to its metric accuracy based on nine real-world test image sequences for which ground truth data were determined. An experimental evaluation of the action recognition stage is conducted using 20 real-world test sequences acquired from different viewpoints in an industrial working environment. We show that our system is able to perform 3D tracking of human body parts and a subsequent recognition of working actions under difficult, realistic conditions. It detects interruptions of the sequence of working actions by entering a safety mode and returns to the regular mode as soon as the working actions continue.