Smartweb: multimodal web services on the road

• Authors: Wolfgang Wahlster
• Venue: Proceedings of the 15th international conference on Multimedia
• Year: 2007

SmartWeb provides a context-aware user interface to webservices, so that it can support the mobile user in differentroles, e.g. as a car driver, a motorbiker, or a pedestrian. Itprovides a symmetric multimodal dialogue system [2] combiningspeech, gesture, haptic and video input with speech, haptic, videoand acoustic output. It goes beyond traditional keyword searchengines like Google by delivering higher quality results that areadapted to the mobile user's current task and situation. In mobilesituations, users don't want to deal with hypertext lists ofretrieved webpages, but simply want an answer to their query. If adesperate driver with a crying and acutely ill child on thebackseat asks SmartWeb "Who is the closest paediatrician?" he needsjust the name and address of the doctor. Based on SmartWeb'sability to combine various web services, the driver can then askSmartWeb a follow-up question about route guidance to the doctor'spractice. One of the innovative features of SmartWeb is that theuser can specify whether he wants a textual or pictorial answer, avideo clip or a sound file as a query result.SmartWeb [1] provides not only an open-domain question answeringmachine but a multimodal web service interface for coherentdialogue, where questions and commands are interpreted according tothe context of the previous conversation. For example, if thedriver of our Mercedes-Benz R-Class test car asks SmartWeb "Whereis the closest Italian restaurant", it will access a web service tofind an appropriate restaurant and show its location on a digitalmap presented on the large dashboard display. The user may continuehis dialog with a command like "Please guide me there with arefuelling stop at the lowest price gas station". In this case,SmartWeb combines a navigation service with a special web servicethat finds low gas prices. SmartWeb includes plan-based compositionmethods for semantic web services, so that complex tasks can becarried out for the mobile user.One version of SmartWeb has been deployed on a BMW motorbikeR1200RT, using a swivel with force feedback integrated in thehandle bar. Similar to the control knob known from the iDriveinterface of BMW automobiles, the biker can rotate the swivel orpush it right or left in order to browse through menus or selectitems displayed by SmartWeb on the large high-resolution screen inthe middle of the cockpit. In combination with these pointingactions, the biker can use speech input over the microphoneintegrated in a Bluetooth helmet to interact with SmartWeb. Themultimodal dialogue system combines visual displays with speech andearcons over the speakers integrated in the helmet and haptic forcefeedback for output generation. For example, the biker can ask forweather forecasts along his planned route. SmartWeb accesseslocation-based web services via the bike's 3G wireless connectionto retrieve the relevant weather forecasts. In addition, SmartWebexploits ad-hoc Wifi connections for vehicle-to-vehiclecommunication based on a local danger warning ontology so that themotorbike driver can be informed of a danger ahead by a car infront of him. For example, a car detecting a large wedge of waterunder its wheels will pass the information wirelessly to the bikefollowing it and SmartWeb will generate the warning "Attention!Risk of aquaplaning 100 meters ahead" using the GPS coordinates ofboth vehicles to compute the distance to the upcoming dangerousarea. Another distinguishing feature of SmartWeb is the generationof adaptive multimodal presentations taking into account thepredicted cognitive load of the biker depending on the drivingspeed and other factors.This keynote presents the anatomy of SmartWeb, itsontology-based information extraction and web service compositiontechnology and explains the distinguishing features of itsmultimodal dialogue and answer engine.