Spatio-temporal discovery: appearance + behavior = agent

  • Authors:
  • Prithwijit Guha;Amitabha Mukerjee;K. S. Venkatesh

  • Affiliations:
  • Department of Electrical Engineering, Indian Institute of Technology, Kanpur, Kanpur, Uttar Pradesh;Department of Computer Science & Engineering, Indian Institute of Technology, Kanpur, Kanpur, Uttar Pradesh;Department of Electrical Engineering, Indian Institute of Technology, Kanpur, Kanpur, Uttar Pradesh

  • Venue:
  • ICVGIP'06 Proceedings of the 5th Indian conference on Computer Vision, Graphics and Image Processing
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Experiments in infant category formation indicate a strong role for temporal continuity and change in perceptual categorization. Computational approaches to model discovery in vision have traditionally focused on static images, with appearance features such as shape playing an important role. In this work, we consider integrating agent behaviors with shape for the purpose of agent discovery. Improved algorithms for video segmentation and tracking under occlusion enable us to construct models that characterize agents in terms of motion and interaction with other objects. We present a preliminary approach for discovering agents based on a combination of appearance and motion histories. Using uncalibrated camera images, we characterize objects discovered in the scene by their shape and motion attributes, and cluster these using agglomerative hierarchical clustering. Even with very simple feature sets, initial results suggest that the approach forms reasonable clusters for diverse categories such as people, and for very distinct clusters (animals), and performs above average on other classes.