I1-[OntoSpace] - Research
Human-Robot Interaction
The goal of this part of the project I1-[OntoSpace] is to explore how humans interact with robots in order to communicate spatially relevant information. In this field, the wider area of variability in spatial reference meets the peculiarities of communicating with artificial interlocutors. Spatial reference can occur with a variety of dialogic strategies, reflecting diverse levels of description varying between goal-based, path-based, and action-based instructions. Goal-based reference builds on principles and preferences in achieving spatial localisation and identification. Results of our experiments show that speakers establish spatial object reference by contrasting the goal object sufficiently from competing objects in a scenario, using a broad range of reference systems underlying a considerable variability in linguistic forms employed for the description. Interpretation problems arise mainly because of underspecification, ambiguity, and many-to-many mappings between linguistic forms and underlying concepts, which furthermore differ between English and German.
In addition to the complexities involved here, preliminary findings indicate that there are different ways of conceptualising both the spatial environment and the artificial communication partner (viz. the robot). That is, the robot may be conceptualized as a tool or as an embodied human-like interlocutor. Also, speakers treat their communication partner as someone who they do not know enough about. Finally, the results show that we can find frequent questions, both directed to the leader of the experiments and to the system itself, regarding those kinds of information speakers consider as prerequisites for formulating their instructions. These clarification questions may cause serious problems for dialogue systems because they can occur basically anytime, and their contents often involve out-of-domain vocabulary. Starting points and a view of the state of the art in dialog modelling are available here.
Here is a demo video with extracts of our Wizard-of-Oz experiments.
Robots
Experiments
No. | Name | Date | Type | Participants, corpus size | Mode | Task | Description | Download |
I1-7 | Rolland_2 | Jan/ Feb 2006 | HRI using Rolland | 51 participants speaking native German and native & non-native English | spoken | Object localisation (static and dynamic); route instruction | Verbal robot reactions; Robot movements are controlled by participant | sample, sound1, sound2 (1.3MB) |
I1-6 | Aibo_2 | Jun/ Jul/ Nov 2004 & Jun/ Jul 2005 | HRI using Aibo, Box | 37 German approx. 25 min. each, 20 English | spoken (& some written) | Object identification, partly sequential; movement | Verbal robot reactions and movement; Comparison between two robots | sample |
I1-5 | Rolland | May 2004 | HRI using Rolland | 27 German approx. 20 min. each, 11 English | spoken | Object localisation (static and dynamic); route instruction | Only input by participant, no reaction by robot; Robot movements are controlled by participant | sample |
I1-4 | Web-experiment | Oct 2003 - Oct 2004 | Web-based corpus collection | >180 native German, ~ 3000 contrib.; >200 native English, ~ 3500 contrib.; >100 nonnative English, ~ 1700 contrib.; approx. 10-15 min. each | typed | Object identification; route instruction | Participants type spatial descriptions according to the given task with respect to various scenarios | sample |
I1-3 | October | Oct/ Nov 2003 | HRI using Aibo, Scorpion, and Pioneer Speedy | Aibo: 11 English, 23 German; Scorpion: 8 English, 22 German; Pioneer: 8 English, 33 German; approx. 20 min. each | typed and spoken | Object identification, partly sequential; movement | Robot sometimes gives scene description; No detailed verbal feedback; Validation of improved implemented computational model of spatial relations; Additional test in HHI (a human is instructed) with 9 participants | transcription movie1 movie2 (4.6M) |
I1-2 | Pioneer Experiment 2 | Jul 2003 | HRI using Pioneer Speedy | 29 German approx. 15 min. each, 482 instructions Complete corpus (xml) Complete extract of all dialogues (html) | typed | Object identification; movement | Robot always gives scene description; Fairly detailed verbal feedback depending on modules; Validation of improved implemented computational model of spatial relations | transcription movie Documentation file (pdf) |
I1-1 | Pioneer Experiment 1 | Feb 2003 | HRI using Pioneer Speedy 25 | German approx. 25 min. each, 1382 instructions Complete corpus (xml) Complete extract of the human-robot interaction data (html) | typed and spoken | Object identification; movement | Robot sometimes gives scene description; No detailed verbal feedback; Validation of improved implemented computational model of spatial relations; Additional test in HHI (a human is instructed) with 9 participants | transcription movie Documentation file (pdf) |
Erlangen | 2003 (elicited at Er-langen) | HRI using Aibo | 51 German approx. 30 min. each | spoken | Object identification, partly sequential; movement | Cooperation with Erlangen research group (Anton Batliner); participants: schoolchildren; Only non-verbal robot reactions (movement) | sample | |
Nats | 2002 (prior to official project start) | HRI using Pioneer Speedy | 21 German approx. 15 min. each, 738 instructions | typed | Object identification; measure | Pre-determined automatic verbal robot reactions | sample | |
WSV | 2000 (prior to official project start) | HRI using Pioneer Giraffe | 15 German, approx. 15 min. each, 467 instructions | typed | Object identification; movement | Exploratory study; Only non-verbal robot reactions (movement); Validation of implemented computational model of spatial relations | sample |