Initial Integration: Ogg That There

Figure 3.2: Netrek Collective Interface: the user uses a deictic gesture to select F0. Cameras on top of the display track the user's movements and a head-mounted microphone is used for speech recognition.

[width=120mm]figs/netrek-pointing

The first version of the Netrek Collective, entitled Ogg That There, is intended to perform in a manner similar to the classic interface demo ``Put That There''[4]. Imperative commands with a subject-verb-object grammar can be issued to individual units. These commands override the robots internal action-selection algorithm, causing the specified action to execute immediately. Objects can either be named explicitly, or referred to with deictic gestures combined with spoken demonstrative pronouns. Figure 3.2 depicts a user selecting a game object with a deictic gesture.

Figure 3.3: Netrek Collective System Diagram. Arrows indicate information flow. The Display module doubles as a database front-end for the Interpretation module so that no modification are needed to the Game Engine.

[width=120mm]figs/netrek

Figure 3.3 illustrates the system architecture of Ogg That There. Thin, solid lines indicate standard socket-based Netrek communications. Thick, dashed lines indicate RPC based communication between our modules. The modules can be distributed across a cluster of machines to allow for future expansion of resource requirements. The following sections give details on the Perception, Interpretation, Display and Robot modules.