Date: 07 November 2014
Location: MD083
Time: 2pm

Abstract:

Virtual (and real) worlds can be populated with artificial characters. Such characters might be companions in a training simulation or video game, engaged with us in a joint task requiring cooperation. The characters share the environment with us, much as other humans share the real world with us, and it would be better if we could talk to them using natural forms of language rather than through pre-selected conversational inputs or through icons on a user interface.

In order to automate language understanding for such artificial characters we need techniques for resolving language references to objects and events. We want to talk to characters about things that exist in the environment or things that have happened. Such techniques need to go beyond co-reference resolution (which has been studied extensively), in which we simply identify which phrases refer to the same object, because we are interested in identifying actual (or virtual) objects. If a character has been told to pick up an object, it needs to know which object to pick up. And, as humans, when we refer to objects we want to use natural phrases like ‘it’, ‘this’, ‘that one over there’, ‘the other one’, ‘the blue one’, etc. rather than using unnatural identifiers over and over again, so our characters need to be able to understand such phrases.

In shared virtual environments objects can become salient to the participants in a conversation (i.e. somehow more prominent) due to their presence in the environment or relevance to a task, so resolving references is about more than the dialogue context (i.e. the form and content of the conversation to date). We also need to take into account the visual and task contexts, both of which could be more important than the dialogue context for a given reference.

In order to implement automated techniques for resolving references in shared virtual environments, therefore, we need to identify which factors lead to objects being salient (or not salient) and how these factors impact upon the resolution of particular forms of phrase. For example, which factors are important for the resolution of the phrase ‘the door’ compared with the phrase ‘this door’? Some phrases don’t explicitly refer to a type at all, such as ‘it’, ‘this’ and ‘that’. How do we determine what those phrases refer to?

In this presentation I will present the techniques I have developed for the resolution of references in shared virtual environments and my results in this field to date. I will also discuss the future of this research.