Recognizing ingredients at cutting process by integrating multimodal features

Authors:
Atsushi Hashimoto;Jin Inoue;Kazuaki Nakamura;Takuya Funatomi;Mayumi Ueda;Yoko Yamakata;Michihiko Minoh
Affiliations:
Kyoto University, Kyoto, Japan;Kyoto University, Kyoto, Japan;Osaka University, Osaka, Japan;Kyoto University, Kyoto, Japan;University of Marketing and Distribution Sciences, Kobe, Japan;Kyoto University, Kyoto, Japan;Kyoto University, Kyoto, Japan
Venue:
Proceedings of the ACM multimedia 2012 workshop on Multimedia for cooking and eating activities
Year:
2012

Citing 9
Cited 1

CounterActive: an interactive cookbook for the kitchen counter

CHI '01 Extended Abstracts on Human Factors in Computing Systems
Authoring of a Mixed Reality Assembly Instructor for Hierarchical Structures

ISMAR '03 Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality
Cooking navi: assistant for daily cooking in kitchen

Proceedings of the 13th annual ACM international conference on Multimedia
Robust Object Detection with Interleaved Categorization and Segmentation

International Journal of Computer Vision
Enabling Calorie-Aware Cooking in a Smart Kitchen

PERSUASIVE '08 Proceedings of the 3rd international conference on Persuasive Technology
Realistic Human Action Recognition with Audio Context

DICTA '10 Proceedings of the 2010 International Conference on Digital Image Computing: Techniques and Applications
Tracking Food Materials with Changing Their Appearance in Food Preparing

ISM '10 Proceedings of the 2010 IEEE International Symposium on Multimedia
Object Recognition Based on Object's Identity for Cooking Recognition Task

ISM '10 Proceedings of the 2010 IEEE International Symposium on Multimedia
Cooking Ingredient Recognition Based on the Load on a Chopping Board during Cutting

ISM '11 Proceedings of the 2011 IEEE International Symposium on Multimedia

Knives are picked before slices are cut: recognition through activity sequence analysis

Proceedings of the 5th international workshop on Multimedia for cooking & eating activities

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a method for recognizing ingredients in food preparing activity. The research for object recognition mainly focuses on only visual information; however, ingredients are difficult to recognize only by visual information because of their limited color variations and larger within-class difference than inter-class difference in shapes. In this paper, we propose a method that involves some physical signals obtained in a cutting process by attaching load and sound sensors to the chopping board. The load may depend on an ingredient's hardness. The sound produced when a knife passes through an ingredient reflects the structure of the ingredient. Hence, these signals are expected to facilitate more precise recognition. We confirmed the effectiveness of the integration of the three modalities (visual, auditory, and load) through experiments in which the developed method was applied to 23 classes of ingredients.