Options
VITRA: Visual Translator
Type
fundamental research project
Start Date
01 April 1985
End Date
31 December 1995
Status
completed
Keywords
Artificial Intelligence
referential semantics for spatial prepositions;
representation and incremental recognition of motion events;
incremental recognition and verbalization of plans
representation and incremental recognition of motion events;
incremental recognition and verbalization of plans
intentions
and plan interactions; listener modelling by means of anticipated imagination; simultaneous natural language descriptions of dynamic imagery; multimodal
incremental route descriptions.
Description
The project VITRA dealt with the relationship between natural language and vision. Experimental studies have been carried out in the way of designing an interface between image-understanding and natural language systems, with the aim of developing systems for the natural language description of real world image sequences.
Leader contributor(s)
Wahlster, Wolfgang
Herzog, Gerd
Member contributor(s)
Partner(s)
Computer Vision (Project V1), IITB, Fraunhofer-Institut für Informations- und Datenverarbeitung, Karlsruhe;
Robotics (Project R1), IPR, Institute for Real-Time Computer Systems and Robotics, Universität Karlsruhe.
Funder(s)
Range
HSG Internal
Range (De)
HSG Intern
Principal
Deutsche Forschungsgemeinschaft (DFG)
Division(s)
Eprints ID
13062
9 results
Now showing
1 - 9 of 9
-
PublicationVITRA GUIDE : Utilisation du Langage Naturel et de Représentations Graphiques pour la Description d'Itinéraires( 1993-05-11)
;Herzog, GerdWazinski, PeterType: conference paper -
PublicationTowards a Computational Semantics of Pathrelations( 1997-01-01)Krueger, A.Type: conference paper
-
PublicationHow Spatial Information Connects Visual Perception and Natural Language Generation in Dynamic Environments: Towards a Computational Model( 1995-01-01)Suppose that you are required to describe a route step-by-step to somebody who does not know the environment. A major question in this context is what kind of spatial information must be integrated in a route description. This task generally refers to two cognitive abilities: Visual perception and natural language. In this domain, a computational model for the generation of incremental route descriptions is presented. Central to this model is a distinction into a visual, a linguistic, and a conceptual-spatial level. Basing on these different levels a software agent, called MOSES, is introduced who moves through a simulated 3D environment from a starting-point to a destination. He selects visuo-spatial information and generates appropriate route descriptions. It is shown how MOSES adopts his linguistic behavior to spatial and temporal constraints. The generation process is based on a corpus of incremental route descriptions which were collected by field experiments. The agent and the 3D environment are entirely implemented.Type: conference paper
-
PublicationSpatial Layout Identification and Incremental Descriptions( 1994-01-01)
;Gapp, Klaus-PeterType: conference paper -
PublicationVITRA GUIDE: Multimodal Route Descriptions for Computer Assisted Vehicle Navigation( 1993-01-01)
;Wazinski, PeterHerzog, GerdWe present a system that generates multimodal route descriptions for computer assisted vehicle navigation. The presentation modes available in Vitra Guide are natural language, maps and perspective views. Depending on situation-specific constraints, e.g., fully specifed route descriptions for pre-trip planning or incremental route descriptions during driving, Vitra Guide allows the coordinated distribution of the route information over the available presentation modes. The coordinated use of the presentation modes is achieved by a plan-based approach. The output for all presentation modes relies on one common 3-dimensional geometric model of the domain.Type: conference paper -
Publication
-
PublicationFrom visual perception to multimodal communication: Incremental route descriptionsIn the last few years in cognitive science there has been a growing interest in the connection between visual perception and natual language. The question of interest is: How ca we discuss what we see? With this question in mind in this article we will look at the area of incremental route descriptions. Here, a speaker step-by-step presents the relevant rout information in a 3D-environment. The speaker must adjust his/her descriptions to the currently visible objects. Two major questions arise in this context: 1. How is visually obtained informaiton used in natural language generation? and 2. How are the modalities coordniated? We will present a computational framework for the interaction of visual perception and natural language descriptions which integrates several processes and representations. Especially discussed is the interaction between the spatial representation and the presentation representation used for natural language descriptions. We have implemented a prototypical version of the proposed model, called MOSES.Type: journal articleJournal: Artificial Intelligence Review JournalVolume: 8Issue: 5/6
-
PublicationVisual Grounding of Route Descriptions in Dynamic Environments( 1995-11-10)
;Baus, Jörg ;Paul, JoachimSrihari, R. K.Type: conference paper -
PublicationMOSES: an example for the modelling of spatial services by means of route description systemIn the last few years in cognitive science there has been a growing interest in the connection between visual perception and natual language. The question of interest is: How ca we discuss what we see? With this question in mind in this article we will look at the area of incremental route descriptions. Here, a speaker step-by-step presents the relevant rout information in a 3D-environment. The speaker must adjust his/her descriptions to the currently visible objects. Two major questions arise in this context: 1. How is visually obtained informaiton used in natural language generation? and 2. How are the modalities coordniated? We will present a computational framework for the interaction of visual perception and natural language descriptions which integrates several processes and representations. Especially discussed is the interaction between the spatial representation and the presentation representation used for natural language descriptions. We have implemented a prototypical version of the proposed model, called MOSES.Type: journal articleJournal: Zeitschrift für SemiotikVolume: 20Issue: 1/2