Figure: User interacting with a movie clip.
When navigating in a virtual environment and meeting both virtual and real characters the use of a keyboard or mouse to give an input to the system can be heavily limiting. First of all, in certain interface tasks using a gesture, rather than clicking or choosing from a menu, can give the user a better feeling of the responsiveness of the system. Also there are tasks that can be given uniquely with gesture input [Kurtenbach and Hulteen1990]. In our system where the user is constantly interacting with a big screen so as to create an immersive type of environment the use of (audio)visual gestures to interact with the environment becomes a key element of the human-computer communication system (see fig. 3).