Subproject 2: Distributed Multimodal Dialogue Component for the Mobile Device

One of the big challenges of this project poses itself at the interface that provides communication between the user and the services of the Semantic Web. The mobile device, a PDA or Smartphone, disposes only of a limited screen size on which the results from the search for information can be displayed. The user can refer to the results multimodally via speech, gestures and other modalities and can thereby refine his inquiry. Here, the user is supposed to be as little constrained as possible. The user is also supposed to be able to choose freely from the modality mix according to the usage context and personal preferences.

The work packages concerning the realization of the multimodal dialogue component take this challenge into account and explore innovative concepts for the interaction in potentially open domains. Work concentrates on interaction relating to the topic of the 2006 FIFA World Cup and to the area of motorbike-related services. The main focus lies here on scientific topics in the areas of modality-fusion and -fission, as well as on multimodal information presentation that can be manipulated by the user via gesture and haptic feedback. The presentation uses innovative visualization technologies to be able to present complex search results in a clear and easy to understand way. From the very beginning the architecture of the system will be defined as distributed and will be composed of high-capacity components that are server-based. In case the connection to the mobile device cannot be established, limited versions of the processing modules take over the interaction. To the user, the transition between online and offline processing should be as transparent as possible. The distributed, scalable architecture of the distributed multimodal dialogue component supports this fluent transition. The work in this Subproject will be closely synchronized with the work in Subproject 3 Embedded Dialogue Component for a Car and the interfaces and processing will also be closely tuned to that Subproject.

© Webmaster
Last modified: Wed Dec 15 16:23:17 CEST