University of St.Gallen
research platform alexandria
search publications
browse publications
by person
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
 
by year

From visual perception to multimodal communication: Incremental route descriptions

abstract In the last few years in cognitive science there has been a growing interest in the connection between visual perception and natual language. The question of interest is: How ca we discuss what we see? With this question in mind in this article we will look at the area of incremental route descriptions. Here, a speaker step-by-step presents the relevant rout information in a 3D-environment. The speaker must adjust his/her descriptions to the currently visible objects. Two major questions arise in this context: 1. How is visually obtained informaiton used in natural language generation? and 2. How are the modalities coordniated? We will present a computational framework for the interaction of visual perception and natural language descriptions which integrates several processes and representations. Especially discussed is the interaction between the spatial representation and the presentation representation used for natural language descriptions. We have implemented a prototypical version of the proposed model, called MOSES.
   
type journal paper
   
keywords
   
project VITRA: Visual Translator
language English
kind of paper journal article
date of appearance 1994
journal Artificial Intelligence Review Journal
publisher Springer (Dordrecht)
ISSN 0269-2821
volume of journal 8
number of issue 5/6
page(s) 68-82
review blind review
   
citation Maass, W. (1994). From visual perception to multimodal communication: Incremental route descriptions. Artificial Intelligence Review Journal, 8(5/6), 68-82.