Food is life. Everyone has to eat. Indeed, if you live to be 80, you will have consumed food more than 87,000 times. Our laboratory is tackling a variety of food related issues through image and video processing, natural language processing, spoken dialogue and chat communication systems, information retrieval, and machine learning. We collaborate with the Aizawa Lab, and students in our laboratory share space and resources with the Aizawa-Yamasaki-Matsui Lab.

The first step to understanding the nutrition we intake is to record the foods eaten. However, the nutritional value of a dish with the same name can vary greatly depending on the recipe. To solve this problem, we are developing RecipeLog, a smartphone application that allows users to record their own recipes with less typing by using AI to predict the user’s input. By linking this app with FoodLog Athl, a food management app developed by Aizawa Lab, users will be able to accurately record their food history at home and understand their nutritional intake. We wish to help not only athletes and people with illnesses, but also everyone understand the food they intake so that they can lead a healthier life.

Cooking is a “creative activity” since it creates new objects from ingredients. Ordinary people tackle a new cuisine just by reading “recipes” (which are a kind of procedural document). Kitchens and their appliances are an active field of IoT, inspiring Smart Kitchen research into “cooking navigation system”, “food storage management”, and “robotic kitchen”. Our Smart Kitchen research focusses on “food and action recognition in cooking by video observation” and “spoken dialogue system to recognize the current cooking situation via conversation and to support it by speech”.

The Web contains countless recipes, but more is not always better. Searching “spaghetti carbonara” gives several thousand recipes, but choosing the best requires deep understanding of language. A recipe is a text representation of a procedural workflow, and we can analyse its characteristic semantic structure using natural language processing. We have implemented a system for translating recipe text into a flowgraph and finding differences between two recipes by node-to-node mapping. Our system can process both Japanese and English recipes.