Platemate: crowdsourcing nutritional analysis from food photographs

Authors:
Jon Noronha;Eric Hysen;Haoqi Zhang;Krzysztof Z. Gajos
Affiliations:
Harvard University & Microsoft, Cambridge, MA, USA;Harvard University & Google, Cambridge, MA, USA;Harvard University, Cambridge, MA, USA;Harvard University, Cambridge, MA, USA
Venue:
Proceedings of the 24th annual ACM symposium on User interface software and technology
Year:
2011

Citing 5
Cited 14

Using Low-Cost Sensing to Support Nutritional Awareness

UbiComp '02 Proceedings of the 4th international conference on Ubiquitous Computing
Exploring iterative and parallel human computation processes

Proceedings of the ACM SIGKDD Workshop on Human Computation
TurKit: human computation algorithms on mechanical turk

UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Soylent: a word processor with a crowd inside

UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Analysis of chewing sounds for dietary monitoring

UbiComp'05 Proceedings of the 7th international conference on Ubiquitous Computing

Augmented perception of satiety: controlling food consumption by changing apparent size of food with augmented reality

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Human computation tasks with global constraints

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
"Honey=sugar" means unhealthy: investigating how people apply knowledge to rate food's healthiness

Proceedings of the 2012 ACM Conference on Ubiquitous Computing
BodyScope: a wearable acoustic sensor for activity recognition

Proceedings of the 2012 ACM Conference on Ubiquitous Computing
How to translate a book within an hour: towards general purpose programmable human computers with CrowdLang

Proceedings of the 3rd Annual ACM Web Science Conference
Crowd saucing: social technology for encouraging healthier eating

interactions
CrowdLang: a programming language for the systematic exploration of human computation systems

SocInfo'12 Proceedings of the 4th international conference on Social Informatics
The future of crowd work

Proceedings of the 2013 conference on Computer supported cooperative work
Crowdsourcing performance evaluations of user interfaces

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Assessing internet video quality using crowdsourcing

Proceedings of the 2nd ACM international workshop on Crowdsourcing for multimedia
Feasibility of identifying eating moments from first-person images leveraging human computation

Proceedings of the 4th International SenseCam & Pervasive Imaging Conference
Simplifying mobile phone food diaries: design and evaluation of a food index-based nutrition diary

Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare
Editorial: Designing for human-food interaction: An introduction to the special issue on 'food and interaction design'

International Journal of Human-Computer Studies
Collectivistic health promotion tools: Accounting for the relationship between culture, food and nutrition

International Journal of Human-Computer Studies

Quantified Score

Hi-index	0.00

Visualization

Abstract

We introduce PlateMate, a system that allows users to take photos of their meals and receive estimates of food intake and composition. Accurate awareness of this information can help people monitor their progress towards dieting goals, but current methods for food logging via self-reporting, expert observation, or algorithmic analysis are time-consuming, expensive, or inaccurate. PlateMate crowdsources nutritional analysis from photographs using Amazon Mechanical Turk, automatically coordinating untrained workers to estimate a meal's calories, fat, carbohydrates, and protein. We present the Management framework for crowdsourcing complex tasks, which supports PlateMate's nutrition analysis workflow. Results of our evaluations show that PlateMate is nearly as accurate as a trained dietitian and easier to use for most users than traditional self-reporting.