Baby talk: Understanding and generating simple image descriptions

  • Authors:
  • G. Kulkarni;V. Premraj;S. Dhar; Siming Li; Yejin Choi;A. C. Berg;T. L. Berg

  • Affiliations:
  • Stony Brook Univ., Stony Brook, NY, USA;Stony Brook Univ., Stony Brook, NY, USA;Stony Brook Univ., Stony Brook, NY, USA;Stony Brook Univ., Stony Brook, NY, USA;Stony Brook Univ., Stony Brook, NY, USA;Stony Brook Univ., Stony Brook, NY, USA;Stony Brook Univ., Stony Brook, NY, USA

  • Venue:
  • CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

We posit that visually descriptive language offers computer vision researchers both information about the world, and information about how people describe the world. The potential benefit from this source is made more significant due to the enormous amount of language data easily available today. We present a system to automatically generate natural language descriptions from images that exploits both statistics gleaned from parsing large quantities of text data and recognition algorithms from computer vision. The system is very effective at producing relevant sentences for images. It also generates descriptions that are notably more true to the specific image content than previous work.